Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9545D200B93 for ; Sat, 17 Sep 2016 06:00:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 93CD2160AD9; Sat, 17 Sep 2016 04:00:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D3045160AC4 for ; Sat, 17 Sep 2016 06:00:24 +0200 (CEST) Received: (qmail 60874 invoked by uid 500); 17 Sep 2016 04:00:24 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 60859 invoked by uid 99); 17 Sep 2016 04:00:23 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 Sep 2016 04:00:23 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id C55612C014C for ; Sat, 17 Sep 2016 04:00:23 +0000 (UTC) Date: Sat, 17 Sep 2016 04:00:23 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-14767) Migrate slow MiniMr tests to faster options MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sat, 17 Sep 2016 04:00:25 -0000 [ https://issues.apache.org/jira/browse/HIVE-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15498114#comment-15498114 ] Hive QA commented on HIVE-14767: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12828901/HIVE-14767.4.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10555 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char] org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1219/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1219/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1219/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12828901 - PreCommit-HIVE-MASTER-Build > Migrate slow MiniMr tests to faster options > ------------------------------------------- > > Key: HIVE-14767 > URL: https://issues.apache.org/jira/browse/HIVE-14767 > Project: Hive > Issue Type: Sub-task > Components: Tests > Affects Versions: 2.2.0 > Reporter: Prasanth Jayachandran > Assignee: Prasanth Jayachandran > Attachments: HIVE-14767.1.patch, HIVE-14767.2.patch, HIVE-14767.3.patch, HIVE-14767.4.patch > > > After analyzing the latest test results https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/ > there are many Minimr tests that are slow individually. There are also some tests that does not have to run in minimr, some tests are testing hdfs related features that can be migrated to MiniLlap (some are already in MiniSpark). Since we have removed hadoop20 now some MiniMr tests can be moved to TestCliDriver. We should keep the absolute minimum number of tests in minimr and move other slow test to minillap (spark already has most of these tests). > These are the individual test runtimes for minimr tests > {code} > QFiles TestMinimrCliDriver elapsed time (Build #1055) > infer_bucket_sort_reducers_power_two.q 356.211 > bucket5.q 257.589 > infer_bucket_sort_bucketed_table.q 251.557 > bucketizedhiveinputformat.q 249.755 > infer_bucket_sort_map_operators.q 249.484 > index_bitmap_auto.q 212.9 > infer_bucket_sort_dyn_part.q 204.401 > skewjoin_onesideskew.q 198.428 > index_bitmap3.q 194.221 > truncate_column_buckets.q 193.966 > auto_sortmerge_join_16.q 171.094 > schemeAuthority.q 170.839 > leftsemijoin_mr.q 150.493 > join_acid_non_acid.q 143.232 > empty_dir_in_table.q 136.463 > bucketmapjoin6.q 129.786 > reduce_deduplicate.q 116.455 > exchgpartition2lel.q 115.678 > bucket_many.q 104.316 > infer_bucket_sort_merge.q 100.976 > quotedid_smb.q 95.801 > external_table_with_space_in_location_path.q 80.265 > schemeAuthority2.q 72.64 > root_dir_external_table.q 72.171 > parallel_orderby.q 71.662 > disable_merge_for_bucketing.q 69.25 > bucket_num_reducers.q 67.618 > remote_script.q 67.029 > groupby2.q 66.72 > bucket4.q 66.02 > udf_using.q 64.985 > file_with_header_footer.q 42.986 > infer_bucket_sort_num_buckets.q 38.795 > scriptfile1.q 35.317 > bucket6.q 34.963 > non_native_window_udf.q 34.129 > join1.q 33.831 > bucketmapjoin7.q 33.71 > bucket_num_reducers2.q 30.645 > list_bucket_dml_10.q 25.751 > insert_dir_distcp.q 25.723 > uber_reduce.q 16.481 > load_hdfs_file_with_space_in_the_name.q 3.899 > table_nonprintable.q 2.097 > load_fs2.q 1.947 > input16_cc.q 0.916 > temp_table_external.q 0.751 > import_exported_table.q 0.508 > scriptfile1_win.q 0.119 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)