Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5348010A6A for ; Mon, 27 Jan 2014 07:19:48 +0000 (UTC) Received: (qmail 35497 invoked by uid 500); 27 Jan 2014 07:19:46 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 35283 invoked by uid 500); 27 Jan 2014 07:19:42 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 34834 invoked by uid 500); 27 Jan 2014 07:19:40 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 34823 invoked by uid 99); 27 Jan 2014 07:19:40 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2014 07:19:40 +0000 Date: Mon, 27 Jan 2014 07:19:40 +0000 (UTC) From: "Hive QA (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-6144) Implement non-staged MapJoin MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882611#comment-13882611 ] Hive QA commented on HIVE-6144: ------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12624965/HIVE-6144.7.patch.txt {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 4959 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name {noformat} Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1039/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1039/console Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12624965 > Implement non-staged MapJoin > ---------------------------- > > Key: HIVE-6144 > URL: https://issues.apache.org/jira/browse/HIVE-6144 > Project: Hive > Issue Type: Improvement > Components: Query Processor > Reporter: Navis > Assignee: Navis > Priority: Minor > Attachments: HIVE-6144.1.patch.txt, HIVE-6144.2.patch.txt, HIVE-6144.3.patch.txt, HIVE-6144.4.patch.txt, HIVE-6144.5.patch.txt, HIVE-6144.6.patch.txt, HIVE-6144.7.patch.txt > > > For map join, all data in small aliases are hashed and stored into temporary file in MapRedLocalTask. But for some aliases without filter or projection, it seemed not necessary to do that. For example. > {noformat} > select a.* from src a join src b on a.key=b.key; > {noformat} > makes plan like this. > {noformat} > STAGE PLANS: > Stage: Stage-4 > Map Reduce Local Work > Alias -> Map Local Tables: > a > Fetch Operator > limit: -1 > Alias -> Map Local Operator Tree: > a > TableScan > alias: a > HashTable Sink Operator > condition expressions: > 0 {key} {value} > 1 > handleSkewJoin: false > keys: > 0 [Column[key]] > 1 [Column[key]] > Position of Big Table: 1 > Stage: Stage-3 > Map Reduce > Alias -> Map Operator Tree: > b > TableScan > alias: b > Map Join Operator > condition map: > Inner Join 0 to 1 > condition expressions: > 0 {key} {value} > 1 > handleSkewJoin: false > keys: > 0 [Column[key]] > 1 [Column[key]] > outputColumnNames: _col0, _col1 > Position of Big Table: 1 > Select Operator > File Output Operator > Local Work: > Map Reduce Local Work > Stage: Stage-0 > Fetch Operator > {noformat} > table src(a) is fetched and stored as-is in MRLocalTask. With this patch, plan can be like below. > {noformat} > Stage: Stage-3 > Map Reduce > Alias -> Map Operator Tree: > b > TableScan > alias: b > Map Join Operator > condition map: > Inner Join 0 to 1 > condition expressions: > 0 {key} {value} > 1 > handleSkewJoin: false > keys: > 0 [Column[key]] > 1 [Column[key]] > outputColumnNames: _col0, _col1 > Position of Big Table: 1 > Select Operator > File Output Operator > Local Work: > Map Reduce Local Work > Alias -> Map Local Tables: > a > Fetch Operator > limit: -1 > Alias -> Map Local Operator Tree: > a > TableScan > alias: a > Has Any Stage Alias: false > Stage: Stage-0 > Fetch Operator > {noformat} -- This message was sent by Atlassian JIRA (v6.1.5#6160)