impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Knupp (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5413) test_seq_writer_hive_compatibility fails on a real cluster because test user lacks write access
Date Sat, 10 Jun 2017 05:46:18 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David Knupp resolved IMPALA-5413.
---------------------------------
    Resolution: Fixed

> test_seq_writer_hive_compatibility fails on a real cluster because test user lacks write
access
> -----------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-5413
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5413
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>    Affects Versions: Impala 2.9.0
>            Reporter: David Knupp
>            Assignee: David Knupp
>            Priority: Critical
>              Labels: remote_cluster_test, test
>
> This test uses the method {{run_stmt_in_hive()}} from {{ImpalaTestSuite}} to perform
a {{select count \( * \) from foo}} query. As written, this method doesn't permit a username
parameter to be passed. If you're running this test on a real cluster, there's no guarantee
that the user who runs the test (which might be 'jenkins', for example, in the case of an
automated test run) has the necessary access on the test cluster to submit a job via Hive.
> The fix should be to add a parameter for username to {{run_stmt_in_hive()}}.
> {noformat}
>  TestTableWriters.test_seq_writer_hive_compatibility[exec_option: {'disable_codegen':
False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes':
0} | table_format: text/none] 
> [gw0] linux2 -- Python 2.7.6 /data/jenkins/workspace/Impala-Quasar-Executor-New/testing/infra/quasar/deploy/executions/runtime/testing/infra/quasar/execution/common_tests/impala/Impala/bin/../infra/python/env/bin/python
> query_test/test_compressed_formats.py:177: in test_seq_writer_hive_compatibility
>     output = self.run_stmt_in_hive('select count(*) from %s' % table_name)
> common/impala_test_suite.py:621: in run_stmt_in_hive
>     raise RuntimeError(stderr)
> E   RuntimeError: SLF4J: Class path contains multiple SLF4J bindings.
> E   SLF4J: Found binding in [jar:file:/data/jenkins/workspace/Impala-Quasar-Executor-New/testing/infra/quasar/deploy/executions/runtime/testing/infra/quasar/execution/common_tests/impala/Impala/thirdparty/hbase-1.2.0-cdh5.12.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> E   SLF4J: Found binding in [jar:file:/data/jenkins/workspace/Impala-Quasar-Executor-New/testing/infra/quasar/deploy/executions/runtime/testing/infra/quasar/execution/common_tests/impala/Impala/thirdparty/hadoop-2.6.0-cdh5.12.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> E   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> E   SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> E   scan complete in 3ms
> E   Connecting to jdbc:hive2://scm-ubuntu1404-mlvdhi-1.vpc.cloudera.com:10000
> E   Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
> E   Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
> E   Transaction isolation: TRANSACTION_REPEATABLE_READ
> E   INFO  : Compiling command(queryId=hive_20170531155353_04ad3f17-cf20-43f7-bb01-78d2feb381e2):
select count(*) from test_seq_writer_hive_compatibility_e3728f35.seq_tbl_NONE_RECORD
> E   INFO  : Semantic Analysis Completed
> E   INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint,
comment:null)], properties:null)
> E   INFO  : Completed compiling command(queryId=hive_20170531155353_04ad3f17-cf20-43f7-bb01-78d2feb381e2);
Time taken: 0.047 seconds
> E   INFO  : Executing command(queryId=hive_20170531155353_04ad3f17-cf20-43f7-bb01-78d2feb381e2):
select count(*) from test_seq_writer_hive_compatibility_e3728f35.seq_tbl_NONE_RECORD
> E   INFO  : Query ID = hive_20170531155353_04ad3f17-cf20-43f7-bb01-78d2feb381e2
> E   INFO  : Total jobs = 1
> E   INFO  : Launching Job 1 out of 1
> E   INFO  : Starting task [Stage-1:MAPRED] in serial mode
> E   INFO  : Number of reduce tasks determined at compile time: 1
> E   INFO  : In order to change the average load for a reducer (in bytes):
> E   INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
> E   INFO  : In order to limit the maximum number of reducers:
> E   INFO  :   set hive.exec.reducers.max=<number>
> E   INFO  : In order to set a constant number of reducers:
> E   INFO  :   set mapreduce.job.reduces=<number>
> E   ERROR : Job Submission failed with exception 'org.apache.hadoop.security.AccessControlException(Permission
denied: user=jenkins, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message