impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amos Bird <amosb...@gmail.com>
Subject Re: test errors in local env
Date Thu, 22 Sep 2016 07:52:20 GMT

Thank you for helping me out :D

> Thanks for keeping this discussion going.
>
> On Sun, Sep 18, 2016 at 8:13 AM, Amos Bird <amosbird@gmail.com> wrote:
>
>>
>> > On Fri, Sep 16, 2016 at 9:06 PM, Amos Bird <amosbird@gmail.com> wrote:
>> >
>> >>
>> >> Hi there,
>> >>
>> >> I followed the wiki
>> >> https://cwiki.apache.org/confluence/display/IMPALA/How+
>> >> to+load+and+run+Impala+tests
>> >> carefully but still have some problems in my local env.
>> >>
>> >> 1. I need to manually execute "hdfs dfs -mkdir
>> /test-warehouse/emptytable"
>> >> to get rid of some fe test error.
>> >>
>> >>
>> > Ideally, you should not have to do this. Could you tell me what errors
>> you
>> > encountered? Sounds like there may be a test or data loading bug we
>> should
>> > fix.
>>
>> The error is :
>>
>> TestLoadData(com.cloudera.impala.analysis.AnalyzeStmtsTest)  Time
>> elapsed: 0.033 sec  <<< FAILURE!
>> java.lang.AssertionError: got error:
>> INPATH location 'hdfs://localhost:20500/test-warehouse/emptytable' does
>> not exist.
>> expected:
>> INPATH location 'hdfs://localhost:20500/test-warehouse/emptytable'
>> contains no visible files.
>>   at org.junit.Assert.fail(Assert.java:88)
>>   at org.junit.Assert.assertTrue(Assert.java:41)
>>   at com.cloudera.impala.common.FrontendTestBase.AnalysisError(
>> FrontendTestBase.java:312)
>>   at com.cloudera.impala.common.FrontendTestBase.AnalysisError(
>> FrontendTestBase.java:292)
>>   at com.cloudera.impala.analysis.AnalyzeStmtsTest.TestLoadData(
>> AnalyzeStmtsTest.java:2860)
>>
>>
> Do you have a table functional.emptytable? If yes, then what location is
> reported in "show create table"?
Query: show create table functional.emptytable
+-------------------------------------------------------------+
| result                                                      |
+-------------------------------------------------------------+
| CREATE EXTERNAL TABLE functional.emptytable (               |
|   field STRING                                              |
| )                                                           |
| PARTITIONED BY (                                            |
|   f2 INT                                                    |
| )                                                           |
| STORED AS TEXTFILE                                          |
| LOCATION 'hdfs://localhost:20500/test-warehouse/emptytable' |
| TBLPROPERTIES ('transient_lastDdlTime'='1464782625')        |
+-------------------------------------------------------------+
Fetched 1 row(s) in 5.51s

> Does the directory exist in HDFS?
No.

>
> You could try to manually reload the table and see if the directory is
> created:
> bin/load-data.py -f -w functional-query --table_names=emptytable
> --table_formats=text/none
After executing this command the directory appears.

>
>>>
>> >> 2. I have authz-policy.ini in HDFS, but I still get authorization
>> errors.
>> >>
>> >> TestSelect[0](com.cloudera.impala.analysis.AuthorizationTest)  Time
>> >> elapsed: 0.333 sec  <<< FAILURE!
>> >> java.lang.AssertionError: got error:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> default.nodb
>> >> expected:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> nodb.alltypes
>> >>   at org.junit.Assert.fail(Assert.java:88)
>> >>   at org.junit.Assert.assertTrue(Assert.java:41)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2220)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2203)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2197)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.TestSelect(
>> >> AuthorizationTest.java:512)
>> >>
>> >> TestSelect[1](com.cloudera.impala.analysis.AuthorizationTest)  Time
>> >> elapsed: 0.324 sec  <<< FAILURE!
>> >> java.lang.AssertionError: got error:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> default.nodb
>> >> expected:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> nodb.alltypes
>> >>   at org.junit.Assert.fail(Assert.java:88)
>> >>   at org.junit.Assert.assertTrue(Assert.java:41)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2220)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2203)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.AuthzError(
>> >> AuthorizationTest.java:2197)
>> >>   at com.cloudera.impala.analysis.AuthorizationTest.TestSelect(
>> >> AuthorizationTest.java:512)
>> >>
>> >>
>> >> Results :
>> >>
>> >> Failed tests:
>> >>   AuthorizationTest.TestSelect:512->AuthzError:2197->
>> >> AuthzError:2203->AuthzError:2220 got error:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> default.nodb
>> >> expected:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> nodb.alltypes
>> >>   AuthorizationTest.TestSelect:512->AuthzError:2197->
>> >> AuthzError:2203->AuthzError:2220 got error:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> default.nodb
>> >> expected:
>> >> User 'amos' does not have privileges to execute 'SELECT' on:
>> nodb.alltypes
>> >>
>> >>
>> >>
>> > Strange. In this test, we register two authorization requests, and it
>> seems
>> > like those are not checked in the expected order. However, that should
>> not
>> > be possible because we store them in a LinkedHashSet.
>> > Could you dig into this a little further to see if you can figure out why
>> > the order is wrong?
>> >
>> > This is where we register the authorization requests:
>> > https://github.com/cloudera/Impala/blob/cdh5-trunk/fe/src/
>> main/java/com/cloudera/impala/analysis/Analyzer.java#L544
>> >
>> > This is where we check the authorization requests:
>> > https://github.com/cloudera/Impala/blob/cdh5-trunk/fe/src/
>> main/java/com/cloudera/impala/analysis/AnalysisContext.java#L391
>> >
>> >
>>
>> I tried directly executing "select 1 from nodb.alltypes" in
>> impala-shell, leading to this error:
>> ERROR: AnalysisException: Could not resolve table reference:
>> 'nodb.alltypes'
>>
>> How can I reproduce the authorization tests in impala-shell so I can
>> debug it?
>>
>>
>>
> FYI, this is actually a known issue and may have something to do with the
> JRE version you are running: https://issues.cloudera.org/browse/IMPALA-3643
> As far as I can tell the bug should be "impossible" because we use a
> LinkedHashSet, but maybe certain JREs do not properly honor the guarantees.
>
> The AuthorizationTests in particular require a non-trivial setup, so I'd
> not recommend trying to debug via the Impala shell.
>
> I'd recommend debugging in one of these ways:
> - Run the test manually via "mvn test -Dtest=AuthorizationTest" from the FE
> directory. Attach debugger and break in TestSelect().
> - Run the JUnit test from an IDE such as Eclipse and then debug the test.
> I'm afraid there is no easy way to just run that single query in our
> current test setup. You will need to run the whole suite, but you can break
> TestSelect() or hack the code in various places to set useful breakpoints.
>
> Hope that helps.

I tried jdk1.8.0_102. Compilation works just fine but AuthorizationTest
fails with ClassNoDefine.
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.008 sec <<< FAILURE!
- in com.cloudera.impala.analysis.AuthorizationTest
initializationError(com.cloudera.impala.analysis.AuthorizationTest)  Time elapsed: 0.006 sec
 <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class com.cloudera.impala.analysis.AuthorizationTest
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
  at org.junit.runners.Parameterized.allParameters(Parameterized.java:280)
  at org.junit.runners.Parameterized.<init>(Parameterized.java:248)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
  at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
  at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
  at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
  at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
  at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
  at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
  at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
  at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
  at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
  at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
  at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

>
>>
>> >>
>> >>
>> >> 3. For end-to-end tests, I encountered two kinds of errors
>> >>
>> >>   a) connection refused.
>> >>
>> >>   SET sync_ddl=False;
>> >> -- executing against localhost:21000
>> >> DROP DATABASE `test_drop_cleans_hdfs_dirs_fdfd4f8` CASCADE;
>> >>
>> >> ___________________ ERROR at setup of TestLoadData.test_load[exec_
>> option:
>> >> {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_
>> threshold':
>> >> 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]
>> >> ___________________
>> >> [gw5] linux2 -- Python 2.6.6 /home/amos/incubator-impala/
>> >> bin/../infra/python/env/bin/python
>> >> metadata/test_load.py:77: in setup_method
>> >>     "{0}/{1}/100101.txt".format(STAGING_PATH, i))
>> >> util/hdfs_util.py:122: in copy
>> >>     data = self.read_file(src)
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> pywebhdfs/webhdfs.py:183:
>> >> in read_file
>> >>     response = requests.get(uri, allow_redirects=True)
>> >> ../infra/python/env/lib/python2.6/site-packages/requests/api.py:69: in
>> get
>> >>     return request('get', url, params=params, **kwargs)
>> >> ../infra/python/env/lib/python2.6/site-packages/requests/api.py:50: in
>> >> request
>> >>     response = session.request(method=method, url=url, **kwargs)
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> requests/sessions.py:465:
>> >> in request
>> >>     resp = self.send(prep, **send_kwargs)
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> requests/sessions.py:594:
>> >> in send
>> >>     history = [resp for resp in gen] if allow_redirects else []
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> requests/sessions.py:196:
>> >> in resolve_redirects
>> >>     **adapter_kwargs
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> requests/sessions.py:573:
>> >> in send
>> >>     r = adapter.send(request, **kwargs)
>> >> ../infra/python/env/lib/python2.6/site-packages/
>> requests/adapters.py:415:
>> >> in send
>> >>     raise ConnectionError(err, request=request)
>> >> E   ConnectionError: ('Connection aborted.', error(111, 'Connection
>> >> refused'))
>> >>
>> >>
>> > The connection refused issue is very bizarre. One thing that I noticed is
>> > that your Python does not seem to match what we use (Python 2.7.3).
>> > Could you re-run infra/python/bootstrap_virtualenv.py and see if you get
>> > the expected version into infra/python/env/local/bin?
>> >
>> > Alternatively, maybe there's a problem with your /etc/hosts? You can try
>> > searching online for WebHdfs and /etc/hosts
>> >
>>
>> well, I find this 'find_py26.py' file under deps. Is it normal?
>>
>
> Yes, that's normal. That file looks Python 2.6 on your system but should
> not be relevant for running tests because we use the Python from our
> virtualenv and not the one on your system.
>
> What's your output when you run "impala-python --version". You should get
> 'Python 2.7.3".

[amos@t450s tests]$ impala-python --version
Python 2.6.6

> Also, what's the Python version on your system? Our virtualenv will use
> Python 2.6 if your system has a Python < 2.6.
> You could try to upgrade your system Python and then
> re-run infra/python/bootstrap_virtualenv.py

My system has python 2.6.6.

>
> Still, theoretically Python 2.6 in the virtual env should work. I think
> it's more likely you are having a connection problem due to a misconfigured
> /etc/hosts.
>
> Are you running the test from a shell that has bin/impala-config.sh and
> bin/set-classpath.sh sourced?
Yes.

>
> To further debug this you could try to specify your namenode address when
> running the test to see whether it is somehow picking up a wrong address:
> cd tests
> ./run-tests.py metadata/test_load.py --namenode_http_address=localhost:50070
>
> And see if that works.

Unfortunately, no.

>
>
>
>> [amos@nobida143 incubator-impala]$ ls infra/python/deps/
>> download_requirements  find_py26.py  pip_download.py  requirements.txt
>> [amos@nobida143 incubator-impala]$ cat infra/python/deps/download_
>> requirements
>> #!/bin/bash
>>
>> # Licensed to the Apache Software Foundation (ASF) under one
>> # or more contributor license agreements.  See the NOTICE file
>> # distributed with this work for additional information
>> # regarding copyright ownership.  The ASF licenses this file
>> # to you under the Apache License, Version 2.0 (the
>> # "License"); you may not use this file except in compliance
>> # with the License.  You may obtain a copy of the License at
>> #
>> #   http://www.apache.org/licenses/LICENSE-2.0
>> #
>> # Unless required by applicable law or agreed to in writing,
>> # software distributed under the License is distributed on an
>> # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>> # KIND, either express or implied.  See the License for the
>> # specific language governing permissions and limitations
>> # under the License.
>>
>> set -euo pipefail
>>
>> DIR="$(dirname "$0")"
>>
>> pushd "$DIR"
>> PY26="$(./find_py26.py)"
>> # Directly download packages listed in requirements.txt, but don't install
>> them.
>> "$PY26" pip_download.py
>> # For virtualenv, other scripts rely on the .tar.gz package (not a .whl
>> package).
>> "$PY26" pip_download.py virtualenv 13.1.0
>> # kudu-python is downloaded separately because pip install attempts to
>> execute a
>> # setup.py subcommand for kudu-python that can fail even if the download
>> succeeds.
>> "$PY26" pip_download.py kudu-python 0.1.1
>> popd
>>
>>
>>
>> >    b) stats not match
>> >>
>> >> [gw4] linux2 -- Python 2.6.6 /home/amos/incubator-impala/
>> >> bin/../infra/python/env/bin/python
>> >> metadata/test_metadata_query_statements.py:67: in test_show_stats
>> >>     self.run_test_case('QueryTest/show-stats', vector, "functional")
>> >> common/impala_test_suite.py:342: in run_test_case
>> >>     self.__verify_results_and_errors(vector, test_section, result,
>> use_db)
>> >> common/impala_test_suite.py:234: in __verify_results_and_errors
>> >>     replace_filenames_with_placeholder)
>> >> common/test_result_verifier.py:398: in verify_raw_results
>> >>     VERIFIER_MAP[verifier](expected, actual)
>> >> common/test_result_verifier.py:231: in verify_query_result_is_equal
>> >>     assert expected_results == actual_results
>> >>
>> >> ...
>> >>
>> >>    -- executing against localhost:21000
>> >> show column stats alltypes_clone;
>> >>
>> >> MainThread: Comparing QueryTestResults (expected vs actual):
>> >> 'bigint_col','BIGINT',10,-1,8,8 == 'bigint_col','BIGINT',10,-1,8,8
>> >> 'bool_col','BOOLEAN',2,-1,1,1 == 'bool_col','BOOLEAN',2,-1,1,1
>> >> 'date_string_col','STRING',736,-1,8,8 == 'date_string_col','STRING',
>> >> 736,-1,8,8
>> >> 'double_col','DOUBLE',-1,-1,8,8 == 'double_col','DOUBLE',-1,-1,8,8
>> >> 'float_col','FLOAT',10,-1,4,4 == 'float_col','FLOAT',10,-1,4,4
>> >> 'id','INT',7505,-1,4,4 == 'id','INT',7505,-1,4,4
>> >> 'int_col','INT',-1,-1,4,4 == 'int_col','INT',-1,-1,4,4
>> >> 'month','INT',12,0,4,4 == 'month','INT',12,0,4,4
>> >> 'smallint_col','SMALLINT',10,-1,2,2 == 'smallint_col','SMALLINT',10,-
>> 1,2,2
>> >> 'string_col','STRING',10,-1,-1,-1 == 'string_col','STRING',10,-1,-1,-1
>> >> 'timestamp_col','TIMESTAMP',7554,-1,16,16 !=
>> 'timestamp_col','TIMESTAMP',
>> >> 7552,-1,16,16
>> >> 'tinyint_col','TINYINT',10,-1,1,1 == 'tinyint_col','TINYINT',10,-1,1,1
>> >> 'year','INT',2,0,4,4 == 'year','INT',2,0,4,4
>> >>
>> >>
>> >> Very strange. Can you do a compute stats on functional.alltypes and
>> > confirm that the NDV for timestamp_col are 7552 in your setup?
>>
>> Yes.
>>
>
> I'll need to ask around for help. I have no idea why this is happening.

Thanks :)

>
>>
>> >
>> >
>> >
>> >> I'm using CentOS 6.8 final. I have no idea what goes wrong. Any help is
>> >> much appreciated!
>> >
>> >
>> >
>> >
>> >>
>> >> Best regards,
>> >> Amos
>> >>
>>
>>


Mime
View raw message