hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1348) Moving inputFileChanged() from ExecMapper to where it is needed
Date Wed, 19 May 2010 18:20:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869246#action_12869246
] 

He Yongqiang commented on HIVE-1348:
------------------------------------

1.
I changed ExecMapperContext to ExecMapperLocalContext because right now it is only used for
map joins. But i can revert it because 'ExecMapperContext ' is more general and can be used
later for other cases.
2. 
Yes. We need to use ExecMapper.memoryMXBean to report memory usage in ExecMapperLocalContext.
This is because it can be used to track memory usage for mapjoin's local work. And once ExecMapper.memoryMXBean
is public, it can also be used in other places.
3. 
will do it.

> Moving inputFileChanged() from ExecMapper to where it is needed
> ---------------------------------------------------------------
>
>                 Key: HIVE-1348
>                 URL: https://issues.apache.org/jira/browse/HIVE-1348
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: He Yongqiang
>         Attachments: hive-1348.1.patch
>
>
> inputFileChanged() is only needed for Bucketed sort merge map join. It should not be
put in ExecMapper.map() where all code paths will hit this function. This function is quite
expensive since JobConf look up is a hash table look up. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message