hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1486) Configuration data should be preserved within the same MapTask
Date Fri, 19 Mar 2010 22:22:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847617#action_12847617

Tom White commented on MAPREDUCE-1486:

This change means that the configuration that the map context uses does have the SKIP_RECORDS
boolean parameter set correctly. More generally, do we know that job isn't updated in other
places - e.g. in the output collectors? Could we pass a single Configuration instance through
to all of these dependents?

> Configuration data should be preserved within the same MapTask
> --------------------------------------------------------------
>                 Key: MAPREDUCE-1486
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1486
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1486.patch
> Map tasks involve a number of Contexts -- at least a TaskAttemptContext and a MapContext.
These context objects contain a Configuration each; when one context is initialized, it initializes
its own Configuration by deep-copying a previous Configuration.
> If one Context instance is used entirely prior to a second, more specific Context then
the second Context should contain the configuration data initialized in the previous Context.
This specifically affects the interaction between an InputFormat and its RecordReader instance(s).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message