hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1348) Moving inputFileChanged() from ExecMapper to where it is needed
Date Wed, 19 May 2010 20:07:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869299#action_12869299
] 

He Yongqiang commented on HIVE-1348:
------------------------------------

1.
We do not want to check the conf  2 times to see if the input file has changed or not. So
that's why the variable  inputFileChanged is used for.  maybe we should give a better name
to 'inputFileChanged()'  (  checkInputFileChanged() ?)
2.
i will change the variable name.
3.
No. they will not change the mapjoin behavior. Those code will only be executed one time for
normal mapjoin.

> Moving inputFileChanged() from ExecMapper to where it is needed
> ---------------------------------------------------------------
>
>                 Key: HIVE-1348
>                 URL: https://issues.apache.org/jira/browse/HIVE-1348
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: He Yongqiang
>         Attachments: hive-1348.1.patch, hive-1348.2.patch
>
>
> inputFileChanged() is only needed for Bucketed sort merge map join. It should not be
put in ExecMapper.map() where all code paths will hit this function. This function is quite
expensive since JobConf look up is a hash table look up. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message