hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1486) Configuration data should be preserved within the same MapTask
Date Fri, 19 Feb 2010 00:08:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835514#action_12835514

Aaron Kimball commented on MAPREDUCE-1486:

I looked at the failing testcases reported by Hudson. None of them reproduced locally:

* TestNodeRefresh seems to be bit by the "Hudson doesn't get the whole classpath" issue that's
been coming up recently. The NoClassDefFoundErrors are really bizarre; does anyone have any
insight into why this has hit Hudson in the last couple weeks?
* TestMapReduceLazyOutput seems to have trouble initializing a MiniDFSCluster and getting
the port it wants; this is unrelated to the patch, as mini-dfs initialization happens outside
any MapContext.
* TestMultipleLevelCaching has a broken test; I filed MAPREDUCE-1508 to address this. Unrelated
failure (looking at the code at TestMultipleLevelCaching.java:122, it's probably a MiniDFS
init problem like TestMapReduceLazyOutput)
* TestJobOutputCommitter also passes locally.

> Configuration data should be preserved within the same MapTask
> --------------------------------------------------------------
>                 Key: MAPREDUCE-1486
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1486
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1486.patch
> Map tasks involve a number of Contexts -- at least a TaskAttemptContext and a MapContext.
These context objects contain a Configuration each; when one context is initialized, it initializes
its own Configuration by deep-copying a previous Configuration.
> If one Context instance is used entirely prior to a second, more specific Context then
the second Context should contain the configuration data initialized in the previous Context.
This specifically affects the interaction between an InputFormat and its RecordReader instance(s).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message