hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14767) Migrate slow MiniMr tests to faster options
Date Sat, 17 Sep 2016 04:00:23 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15498114#comment-15498114
] 

Hive QA commented on HIVE-14767:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12828901/HIVE-14767.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10555 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1219/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1219/console
Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1219/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12828901 - PreCommit-HIVE-MASTER-Build

> Migrate slow MiniMr tests to faster options
> -------------------------------------------
>
>                 Key: HIVE-14767
>                 URL: https://issues.apache.org/jira/browse/HIVE-14767
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14767.1.patch, HIVE-14767.2.patch, HIVE-14767.3.patch, HIVE-14767.4.patch
>
>
> After analyzing the latest test results https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/
> there are many Minimr tests that are slow individually. There are also some tests that
does not have to run in minimr, some tests are testing hdfs related features that can be migrated
to MiniLlap (some are already in MiniSpark). Since we have removed hadoop20 now some MiniMr
tests can be moved to TestCliDriver. We should keep the absolute minimum number of tests in
minimr and move other slow test to minillap (spark already has most of these tests).
> These are the individual test runtimes for minimr tests
> {code}
> QFiles	TestMinimrCliDriver elapsed time (Build #1055)
> infer_bucket_sort_reducers_power_two.q	356.211
> bucket5.q	257.589
> infer_bucket_sort_bucketed_table.q	251.557
> bucketizedhiveinputformat.q	249.755
> infer_bucket_sort_map_operators.q	249.484
> index_bitmap_auto.q	212.9
> infer_bucket_sort_dyn_part.q	204.401
> skewjoin_onesideskew.q	198.428
> index_bitmap3.q	194.221
> truncate_column_buckets.q	193.966
> auto_sortmerge_join_16.q	171.094
> schemeAuthority.q	170.839
> leftsemijoin_mr.q	150.493
> join_acid_non_acid.q	143.232
> empty_dir_in_table.q	136.463
> bucketmapjoin6.q	129.786
> reduce_deduplicate.q	116.455
> exchgpartition2lel.q	115.678
> bucket_many.q	104.316
> infer_bucket_sort_merge.q	100.976
> quotedid_smb.q	95.801
> external_table_with_space_in_location_path.q	80.265
> schemeAuthority2.q	72.64
> root_dir_external_table.q	72.171
> parallel_orderby.q	71.662
> disable_merge_for_bucketing.q	69.25
> bucket_num_reducers.q	67.618
> remote_script.q	67.029
> groupby2.q	66.72
> bucket4.q	66.02
> udf_using.q	64.985
> file_with_header_footer.q	42.986
> infer_bucket_sort_num_buckets.q	38.795
> scriptfile1.q	35.317
> bucket6.q	34.963
> non_native_window_udf.q	34.129
> join1.q	33.831
> bucketmapjoin7.q	33.71
> bucket_num_reducers2.q	30.645
> list_bucket_dml_10.q	25.751
> insert_dir_distcp.q	25.723
> uber_reduce.q	16.481
> load_hdfs_file_with_space_in_the_name.q	3.899
> table_nonprintable.q	2.097
> load_fs2.q	1.947
> input16_cc.q	0.916
> temp_table_external.q	0.751
> import_exported_table.q	0.508
> scriptfile1_win.q	0.119
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message