Mailing-List: contact hive-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-dev@hadoop.apache.org
Message-ID: <18086162.26661274470337754.JavaMail.jira@thor>
Date: Fri, 21 May 2010 15:32:17 -0400 (EDT)
From: "He Yongqiang (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Subject: [jira] Commented: (HIVE-1348) Moving inputFileChanged() from
 ExecMapper to where it is needed
In-Reply-To: <26688146.110991274205042778.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870124#action_12870124 ] 

He Yongqiang commented on HIVE-1348:
------------------------------------

Yeah, I agree. However, the event of input file changed can only be emitted one time. If we store it in one operator, other operators may miss this event. So that's why we maintain a kind of global variable for all operators.  

> Moving inputFileChanged() from ExecMapper to where it is needed
> ---------------------------------------------------------------
>
>                 Key: HIVE-1348
>                 URL: https://issues.apache.org/jira/browse/HIVE-1348
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: He Yongqiang
>         Attachments: hive-1348.1.patch, hive-1348.2.patch, hive-1348.3.patch
>
>
> inputFileChanged() is only needed for Bucketed sort merge map join. It should not be put in ExecMapper.map() where all code paths will hit this function. This function is quite expensive since JobConf look up is a hash table look up. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.