hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14877) Move slow CliDriver tests to MiniLlap
Date Fri, 07 Oct 2016 19:16:20 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-14877:
-----------------------------------------
    Attachment: HIVE-14877.5.patch

Explicitly setting the stats to avoid flakiness in the test. 

> Move slow CliDriver tests to MiniLlap
> -------------------------------------
>
>                 Key: HIVE-14877
>                 URL: https://issues.apache.org/jira/browse/HIVE-14877
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14877.1.patch, HIVE-14877.2.patch, HIVE-14877.3.patch, HIVE-14877.4.patch,
HIVE-14877.5.patch
>
>
> When analyzing the test runtimes, there are many CliDriver tests that shows up as stragglers
and are slow. Most of these tests are not really testing the execution engine. For example
special_character_in_tabnames_1.q is the slowest test case that takes 419s in CliDriver but
only 62s in MiniLlap. Similarly there are many test cases that can benefit from fast runtimes.
We should consider moving the tests that are not testing the execution engine to MiniLlap
(assuming it provides significant performance benefit).
> Here is the list of top 100 slow tests based on build #1055
> ||QFiles||TestCliDriver elapsed time||
> |special_character_in_tabnames_1.q|419.229|
> |unionDistinct_1.q|278.583|
> |vector_leftsemi_mapjoin.q|232.313|
> |join_filters.q|172.436|
> |escape2.q|167.503|
> |archive_excludeHadoop20.q|163.522|
> |escape1.q|130.217|
> |lineage3.q|110.935|
> |insert_into_with_schema.q|107.345|
> |auto_join_filters.q|104.331|
> |windowing.q|99.622|
> |index_compact_binary_search.q|97.637|
> |cbo_rp_windowing_2.q|95.108|
> |vectorized_ptf.q|93.397|
> |dynpart_sort_optimization_acid.q|91.831|
> |partition_multilevels.q|90.392|
> |ptf.q|89.115|
> |sample_islocalmode_hook.q|88.293|
> |udaf_collect_set_2.q|84.725|
> |skewjoin.q|84.588|
> |lineage2.q|84.187|
> |correlationoptimizer1.q|80.367|
> |dynpart_sort_optimization.q|77.07|
> |orc_ppd_decimal.q|75.523|
> |orc_ppd_schema_evol_3a.q|75.352|
> |groupby_sort_skew_1_23.q|75.342|
> |cbo_rp_lineage2.q|75.283|
> |parquet_ppd_decimal.q|74.063|
> |sample_islocalmode_hook_use_metadata.q|73.988|
> |orc_analyze.q|73.803|
> |join_nulls.q|72.417|
> |semijoin.q|70.403|
> |correlationoptimizer6.q|69.151|
> |table_access_keys_stats.q|68.699|
> |autoColumnStats_2.q|68.632|
> |cbo_join.q|68.325|
> |cbo_rp_join.q|68.317|
> |sample10.q|64.513|
> |mergejoin.q|63.647|
> |multi_insert_move_tasks_share_dependencies.q|62.079|
> |union_view.q|61.772|
> |autoColumnStats_1.q|61.246|
> |groupby_sort_1_23.q|61.129|
> |pcr.q|59.546|
> |vectorization_short_regress.q|58.775|
> |auto_sortmerge_join_9.q|58.3|
> |correlationoptimizer2.q|56.591|
> |alter_merge_stats_orc.q|55.202|
> |vector_join30.q|54.85|
> |selectDistinctStar.q|53.981|
> |vector_decimal_udf.q|53.879|
> |auto_join30.q|53.762|
> |subquery_notin.q|52.879|
> |cbo_rp_subq_not_in.q|52.609|
> |cbo_rp_gby.q|51.866|
> |cbo_subq_not_in.q|51.672|
> |cbo_gby.q|50.361|
> |infer_bucket_sort.q|49.158|
> |ptf_streaming.q|48.484|
> |join_1to1.q|48.268|
> |load_dyn_part5.q|47.796|
> |limit_join_transpose.q|47.517|
> |ppd_windowing2.q|47.318|
> |dynpart_sort_opt_vectorization.q|47.208|
> |vector_number_compare_projection.q|47.024|
> |correlationoptimizer4.q|45.472|
> |orc_ppd_date.q|45.19|
> |global_limit.q|44.438|
> |union_top_level.q|44.229|
> |llap_partitioned.q|44.139|
> |orc_ppd_timestamp.q|43.617|
> |parquet_ppd_date.q|43.539|
> |multiMapJoin2.q|43.036|
> |parquet_ppd_timestamp.q|42.665|
> |vector_partitioned_date_time.q|42.511|
> |auto_sortmerge_join_8.q|42.377|
> |create_view.q|42.23|
> |windowing_windowspec2.q|42.202|
> |multiMapJoin1.q|41.176|
> |vector_decimal_2.q|41.026|
> |bucket_groupby.q|40.565|
> |rcfile_merge2.q|39.782|
> |index_compact_2.q|39.765|
> |join_nullsafe.q|39.698|
> |vector_join_filters.q|39.343|
> |cbo_rp_auto_join1.q|39.308|
> |vector_auto_smb_mapjoin_14.q|39.17|
> |vector_udf1.q|38.988|
> |rcfile_createas1.q|38.932|
> |cbo_rp_semijoin.q|38.675|
> |auto_join_nulls.q|38.519|
> |cbo_rp_unionDistinct_2.q|37.815|
> |union_remove_26.q|37.672|
> |rcfile_merge3.q|37.373|
> |rcfile_merge4.q|37.194|
> |bucketsortoptimize_insert_2.q|37.187|
> |cbo_limit.q|37.038|
> |auto_sortmerge_join_6.q|36.663|
> |join43.q|36.656|



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message