hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-19637) Add slow test report script to testutils
Date Mon, 21 May 2018 19:05:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-19637:
-----------------------------------------
    Attachment:     (was: HIVE-19637.1.patch)

> Add slow test report script to testutils
> ----------------------------------------
>
>                 Key: HIVE-19637
>                 URL: https://issues.apache.org/jira/browse/HIVE-19637
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Test
>    Affects Versions: 3.1.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Major
>
> Wrote the attached utility script to find top K slow tests from precommit test url. Would
like to get that committed to testutils so that its useful for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
> ##########################################################################################################
> ██████████████████████████████████████████████████
 20806  TestCliDriver
> ███████████████████████               
              9601  TestMiniLlapLocalCliDriver
> ███████████████████                           
      8210  TestSparkCliDriver
> ██████                                               2744  TestMinimrCliDriver
> █████                                                2262  TestEncryptedHDFSCliDriver
> ████                                                 2021  TestMiniSparkOnYarnCliDriver
> ████                                                 1808  TestHiveCli
> ███                                                  1566  TestMiniLlapCliDriver
> ███                                                  1345  TestReplicationScenarios
> ██                                                   1238  TestMiniDruidCliDriver
> ██                                                    940  TestNegativeCliDriver
> ██                                                    865  TestHBaseCliDriver
> █                                                     681  TestMiniTezCliDriver
> █                                                     555  TestTxnCommands2WithSplitUpdateAndVectorization
> █                                                     543  TestCompactor
> █                                                     528  TestTxnCommands2
>                                                       378  TestStreaming
>                                                       374  TestBlobstoreCliDriver
>                                                       328  TestNegativeMinimrCliDriver
>                                                       302  TestTxnCommandsWithSplitUpdateAndVectorization
>                                                       301  TestHCatClient
>                                                       299  TestTxnCommands
>                                                       261  TestTxnLoadData
>                                                       258  TestAcidOnTez
>                                                       240  TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
> ###############################################################################################################################################
> ██████████████████████████████████████████████████
 680  TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █████████████████████████████████████████████
      623  TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███████████████████████████████
                    429  TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███████████████████████████   
                     374  TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███████████████████████████   
                     374  TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> ████████████████████████            
               330  TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █████████████████                                 
 238  TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> ████████████████                                    227
 TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███████████████                                     214
 TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███████████████                                     211
 TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███████████████                                     210
 TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]
> ███████████████                                     206
 TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]
> ██████████████                                      202 
TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]
> ██████████████                                      198 
TestCliDriver_testCliDriver[typechangetest]
> ████████████                                        172  TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]
> ████████████                                        164  TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]
> ███████████                                         158  TestCliDriver_testCliDriver[mm_all]
> ███████████                                         155  TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]
> ██████████                                          145  TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
> ██████████                                          141  TestMiniLlapCliDriver_testCliDriver[mm_all]
> ██████████                                          140  TestMiniDruidCliDriver_testCliDriver[druidmini_mv]
> ██████████                                          137  TestSparkCliDriver_testCliDriver[auto_join_filters]
> █████████                                           135  TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]
> █████████                                           124  TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]
> ████████                                            121  TestCliDriver_testCliDriver[type_change_test_int_vectorized]
> {code}
> {code:title=text mode}
> $ python gen-report.py -b 11102
> Processing 1073 test xml reports from http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time: 73882.661 seconds]
> TestCliDriver	20805.579
> TestMiniLlapLocalCliDriver	9601.362
> TestSparkCliDriver	8210.062
> TestMinimrCliDriver	2743.746
> TestEncryptedHDFSCliDriver	2261.866
> TestMiniSparkOnYarnCliDriver	2021.468
> TestHiveCli	1807.56
> TestMiniLlapCliDriver	1565.858
> TestReplicationScenarios	1345.344
> TestMiniDruidCliDriver	1237.776
> TestNegativeCliDriver	940.321
> TestHBaseCliDriver	864.707
> TestMiniTezCliDriver	681.457
> TestTxnCommands2WithSplitUpdateAndVectorization	555.382
> TestCompactor	543.173
> TestTxnCommands2	527.84
> TestStreaming	378.325
> TestBlobstoreCliDriver	374.134
> TestNegativeMinimrCliDriver	328.128
> TestTxnCommandsWithSplitUpdateAndVectorization	301.718
> TestHCatClient	301.324
> TestTxnCommands	298.694
> TestTxnLoadData	260.841
> TestAcidOnTez	258.262
> TestHBaseNegativeCliDriver	240.198
> Top 25 testcases in terms of execution time (in seconds).. [Total time: 63102.607 seconds]
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]	680.326
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]	623.404
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]	429.358
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]	374.491
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]	374.164
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]	329.945
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]	238.5
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]	226.773
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]	214.09
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]	210.803
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join5]	209.894
> TestMinimrCliDriver_testCliDriver[bucket_num_reducers_acid]	206.339
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_merge]	201.754
> TestCliDriver_testCliDriver[typechangetest]	198.371
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_drop_table]	172.267
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_num_buckets]	163.617
> TestCliDriver_testCliDriver[mm_all]	158.401
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_dynamic_partition_pruning]	155.255
> TestMiniSparkOnYarnCliDriver_testCliDriver[spark_vectorized_dynamic_partition_pruning]
145.481
> TestMiniLlapCliDriver_testCliDriver[mm_all]	141.369
> TestMiniDruidCliDriver_testCliDriver[druidmini_mv]	139.815
> TestSparkCliDriver_testCliDriver[auto_join_filters]	137.391
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join3]	135.227
> TestMiniSparkOnYarnCliDriver_testCliDriver[vector_outer_join3]	124.469
> TestCliDriver_testCliDriver[type_change_test_int_vectorized]	120.85
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message