hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14767) Migrate slow MiniMr tests to faster options
Date Thu, 15 Sep 2016 20:58:20 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-14767:
-----------------------------------------
    Status: Patch Available  (was: Open)

> Migrate slow MiniMr tests to faster options
> -------------------------------------------
>
>                 Key: HIVE-14767
>                 URL: https://issues.apache.org/jira/browse/HIVE-14767
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14767.1.patch
>
>
> After analyzing the latest test results https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/
> there are many Minimr tests that are slow individually. There are also some tests that
does not have to run in minimr, some tests are testing hdfs related features that can be migrated
to MiniLlap (some are already in MiniSpark). Since we have removed hadoop20 now some MiniMr
tests can be moved to TestCliDriver. We should keep the absolute minimum number of tests in
minimr and move other slow test to minillap (spark already has most of these tests).
> These are the individual test runtimes for minimr tests
> {code}
> QFiles	TestMinimrCliDriver elapsed time (Build #1055)
> infer_bucket_sort_reducers_power_two.q	356.211
> bucket5.q	257.589
> infer_bucket_sort_bucketed_table.q	251.557
> bucketizedhiveinputformat.q	249.755
> infer_bucket_sort_map_operators.q	249.484
> index_bitmap_auto.q	212.9
> infer_bucket_sort_dyn_part.q	204.401
> skewjoin_onesideskew.q	198.428
> index_bitmap3.q	194.221
> truncate_column_buckets.q	193.966
> auto_sortmerge_join_16.q	171.094
> schemeAuthority.q	170.839
> leftsemijoin_mr.q	150.493
> join_acid_non_acid.q	143.232
> empty_dir_in_table.q	136.463
> bucketmapjoin6.q	129.786
> reduce_deduplicate.q	116.455
> exchgpartition2lel.q	115.678
> bucket_many.q	104.316
> infer_bucket_sort_merge.q	100.976
> quotedid_smb.q	95.801
> external_table_with_space_in_location_path.q	80.265
> schemeAuthority2.q	72.64
> root_dir_external_table.q	72.171
> parallel_orderby.q	71.662
> disable_merge_for_bucketing.q	69.25
> bucket_num_reducers.q	67.618
> remote_script.q	67.029
> groupby2.q	66.72
> bucket4.q	66.02
> udf_using.q	64.985
> file_with_header_footer.q	42.986
> infer_bucket_sort_num_buckets.q	38.795
> scriptfile1.q	35.317
> bucket6.q	34.963
> non_native_window_udf.q	34.129
> join1.q	33.831
> bucketmapjoin7.q	33.71
> bucket_num_reducers2.q	30.645
> list_bucket_dml_10.q	25.751
> insert_dir_distcp.q	25.723
> uber_reduce.q	16.481
> load_hdfs_file_with_space_in_the_name.q	3.899
> table_nonprintable.q	2.097
> load_fs2.q	1.947
> input16_cc.q	0.916
> temp_table_external.q	0.751
> import_exported_table.q	0.508
> scriptfile1_win.q	0.119
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message