Mailing-List: contact hive-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-dev@hadoop.apache.org
Message-ID: <10544046.63451283145738855.JavaMail.jira@thor>
Date: Mon, 30 Aug 2010 01:22:18 -0400 (EDT)
From: "Amareshwari Sriramadasu (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Subject: [jira] Commented: (HIVE-1605) regression and improvements in
 handling NULLs in joins
In-Reply-To: <8530693.38691282940093318.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904085#action_12904085 ] 

Amareshwari Sriramadasu commented on HIVE-1605:
-----------------------------------------------

Ning, Thanks for looking into this. A couple of minor comments:
* Patch has many debug logs and commented code. do you want to remove them?
* Do you want to remove hasAllNulls method from AbstractMapJoinOperator.java?


> regression and improvements in handling NULLs in joins
> ------------------------------------------------------
>
>                 Key: HIVE-1605
>                 URL: https://issues.apache.org/jira/browse/HIVE-1605
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: Ning Zhang
>         Attachments: HIVE-1605.patch
>
>
> There are regressions in sort-merge map join after HIVE-741. There are a lot of OOM exceptions in SMBMapJoinOperator. This caused by the HashMap maintained for each key to remember whether it is NULL. This takes too much memory when the tables are large. 
> A second issu is in handling NULLs if the join keys are more than 1 column. This appears in regular MapJoin as well as SMBMapJoin. The code only checks if all the columns are NULL. It should return false in match if any joined value is NULL. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.