hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Friedrich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7953) Investigate query failures (2)
Date Sun, 12 Oct 2014 06:03:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168541#comment-14168541
] 

Thomas Friedrich commented on HIVE-7953:
----------------------------------------

The tests
dbtxnmgr_compact1
dbtxnmgr_compact2
dbtxnmgr_compact3
dbtxnmgr_ddl1
dbtxnmgr_query2
dbtxnmgr_query5
all fail with the same error:
FAILED: LockException [Error 10280]: Error communicating with the metastore

I did some investigation and found that the metastore tables are not initialized properly.
The underlying exception is: 
org.apache.thrift.TException: MetaException(message:Unable to select from transaction database,
java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does not exist.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedStatement.executeQuery(Unknown Source)
	at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
	at org.apache.hadoop.hive.metastore.txn.TxnHandler.timeOutTxns(TxnHandler.java:1625)
	at org.apache.hadoop.hive.metastore.txn.TxnHandler.getOpenTxns(TxnHandler.java:197)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5273)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

When I debugged, I found that in TxnHandler.java, the check HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEST)
in method checkQFileTestHack() returns false:
  private void checkQFileTestHack() {
    boolean hackOn = HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEST) ||
        HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEZ_TEST);

If that is false, it won't call prepDb() which creates the required tables. According to HiveConf.java,
HIVE_IN_TEST should be true in test mode, not sure why it's not the case here.



> Investigate query failures (2)
> ------------------------------
>
>                 Key: HIVE-7953
>                 URL: https://issues.apache.org/jira/browse/HIVE-7953
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Brock Noland
>            Assignee: Thomas Friedrich
>
> I ran all q-file tests and the following failed with an exception:
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-SPARK-ALL-TESTS-Build/lastCompletedBuild/testReport/
> we don't necessary want to run all these tests as part of the spark tests, but we should
understand why they failed with an exception. This JIRA is to look into these failures and
document them with one of:
> * New JIRA
> * Covered under existing JIRA
> * More investigation required
> Tests:
> {noformat}
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_temp_table_external	0.33
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucket_num_reducers	4.3
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_2	11
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
0.65 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_4
4.7 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_7
2.8 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_2
5.5 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_position	1.5 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_18_part_external	2.4
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_6	11
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_smb_mapjoin_11	5.1 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_8
10 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join	5.4 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_empty_dyn_part	0.81
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact1	0.31 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_ddl1	0.26 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query2	0.73 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_3	8.5
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query5	0.34 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_rcfile_bigdata	0.93 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_multi_single_reducer
6.3 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact3	2.4 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact2	0.56 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_partscan_1_23	3.1
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_list_bucket_dml_2	4.3 sec
2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_15_external_part	3.2
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_16_part_external	2.8
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_17_part_managed	3.4
sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_20_part_managed_location
3.3 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_19_00_part_external_location
6.9 sec	2
>  org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_external_table_with_space_in_location_path
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message