Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B2B1910F8D for ; Thu, 21 Nov 2013 06:39:37 +0000 (UTC) Received: (qmail 15463 invoked by uid 500); 21 Nov 2013 06:39:36 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 15384 invoked by uid 500); 21 Nov 2013 06:39:35 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 15363 invoked by uid 500); 21 Nov 2013 06:39:35 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 15358 invoked by uid 99); 21 Nov 2013 06:39:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2013 06:39:35 +0000 Date: Thu, 21 Nov 2013 06:39:35 +0000 (UTC) From: "Hive QA (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-5849) Improve the stats of operators based on heuristics in the absence of any column statistics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828537#comment-13828537 ] Hive QA commented on HIVE-5849: ------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12615014/HIVE-5849.5.patch {color:red}ERROR:{color} -1 due to 114 failed/errored test(s), 4680 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynamic_partition_skip_default org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_rearrange org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_auto org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input39_hadoop20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join38 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join40 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_cp org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_lateral_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_gby_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_extended org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_json_tuple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_parse_url_tuple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_left_outer_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_shufflejoin org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_map_operators {noformat} Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/382/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/382/console Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 114 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12615014 > Improve the stats of operators based on heuristics in the absence of any column statistics > ------------------------------------------------------------------------------------------ > > Key: HIVE-5849 > URL: https://issues.apache.org/jira/browse/HIVE-5849 > Project: Hive > Issue Type: Sub-task > Components: Query Processor, Statistics > Reporter: Prasanth J > Assignee: Prasanth J > Fix For: 0.13.0 > > Attachments: HIVE-5849.1.patch.txt, HIVE-5849.2.patch.txt, HIVE-5849.3.patch, HIVE-5849.3.patch.txt, HIVE-5849.4.javaonly.patch, HIVE-5849.5.patch > > > In the absence of any column statistics, operators will simply use the statistics from its parents. It is useful to apply some heuristics to update basic statistics (number of rows and data size) in the absence of any column statistics. This will be worst case scenario. -- This message was sent by Atlassian JIRA (v6.1#6144)