hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19589) NOOP jira to see which tests are flaky on HiveQA
Date Fri, 18 May 2018 18:39:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481021#comment-16481021
] 

Jesus Camacho Rodriguez commented on HIVE-19589:
------------------------------------------------

An initial list of tests that I have detected flaky for these runs and other runs:
{noformat}
 org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[4]
 org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal
 org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.org.apache.hive.jdbc.TestTriggersTezSessionPoolManager
 org.apache.hive.jdbc.TestTriggersWorkloadManager.org.apache.hive.jdbc.TestTriggersWorkloadManager
 org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier2
 org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
 org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
 org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversion02
 org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingMetaStore
 org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeUnset
 org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedNoAutoPurge
 org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateUnsetAutoPurge
 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_move_tasks_share_dependencies]
 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
 org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_stats]
{noformat}

A few notes:
- For {{union_stats}}, there is work going on in HIVE-19326. We can disable it till it gets
in.
- For {{TestReoptimization}}, HIVE-19574. We can also disable that one till that gets fixed.
- {{TestAutoPurgeTables}} has been failing quite a lot, we added the retry annotation, but
it does not seem to help. Candidate to disable for the time being?
- Same for triggers tests, usually with following error:
{noformat}
Error Message
java.net.BindException: Address already in use
{noformat}
- For some tests, mocking objects could fix a great deal. For instance, {{testQuotedIdentifier2}}
failed with the following stacktrace:
{code}
org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Column stats doesn't exist for
db=default temp table=acidTbl2
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.deleteTempTableColumnStatsForTable(SessionHiveMetaStoreClient.java:795)
~[classes/:?]
	at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.drop_table_with_environment_context(SessionHiveMetaStoreClient.java:135)
[classes/:?]
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1268)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1222)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1203)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
	at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
[hive-standalone-metastore-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
	at com.sun.proxy.$Proxy39.dropTable(Unknown Source) [?:?]
	at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:1033) [classes/:?]
	at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:970) [classes/:?]
	at org.apache.hadoop.hive.ql.exec.DDLTask.dropTable(DDLTask.java:4736) [classes/:?]
	at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:4590) [classes/:?]
	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:438) [classes/:?]
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) [classes/:?]
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) [classes/:?]
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2482) [classes/:?]
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2149) [classes/:?]
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1826) [classes/:?]
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1569) [classes/:?]
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1558) [classes/:?]
	at org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.runStatementOnDriver(TxnCommandsBaseForTests.java:152)
[test-classes/:?]
	at org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.dropTables(TxnCommandsBaseForTests.java:111)
[test-classes/:?]
	at org.apache.hadoop.hive.ql.TxnCommandsBaseForTests.tearDown(TxnCommandsBaseForTests.java:118)
[test-classes/:?]
	at sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source) ~[?:?]
{code}
This is when we are tearing down the test, hence maybe we could even ignore the failure. But
apart from that, I wonder whether we needed such a complex setup in the first place to test
quoted identifiers? Maybe such test could be rewritten to use a mock implementation for some
of those classes.

[~sershe], let's get on top of them and get more consistent green runs from today. How do
you want to follow-up on this?

> NOOP jira to see which tests are flaky on HiveQA
> ------------------------------------------------
>
>                 Key: HIVE-19589
>                 URL: https://issues.apache.org/jira/browse/HIVE-19589
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>            Priority: Major
>         Attachments: HIVE-19589.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message