hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1486) Configuration data should be preserved within the same MapTask
Date Fri, 19 Feb 2010 00:08:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835514#action_12835514
] 

Aaron Kimball commented on MAPREDUCE-1486:
------------------------------------------

I looked at the failing testcases reported by Hudson. None of them reproduced locally:

* TestNodeRefresh seems to be bit by the "Hudson doesn't get the whole classpath" issue that's
been coming up recently. The NoClassDefFoundErrors are really bizarre; does anyone have any
insight into why this has hit Hudson in the last couple weeks?
* TestMapReduceLazyOutput seems to have trouble initializing a MiniDFSCluster and getting
the port it wants; this is unrelated to the patch, as mini-dfs initialization happens outside
any MapContext.
* TestMultipleLevelCaching has a broken test; I filed MAPREDUCE-1508 to address this. Unrelated
failure (looking at the code at TestMultipleLevelCaching.java:122, it's probably a MiniDFS
init problem like TestMapReduceLazyOutput)
* TestJobOutputCommitter also passes locally.

> Configuration data should be preserved within the same MapTask
> --------------------------------------------------------------
>
>                 Key: MAPREDUCE-1486
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1486
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1486.patch
>
>
> Map tasks involve a number of Contexts -- at least a TaskAttemptContext and a MapContext.
These context objects contain a Configuration each; when one context is initialized, it initializes
its own Configuration by deep-copying a previous Configuration.
> If one Context instance is used entirely prior to a second, more specific Context then
the second Context should contain the configuration data initialized in the previous Context.
This specifically affects the interaction between an InputFormat and its RecordReader instance(s).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message