hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4897) Using both MultipleInputs.addInputPath methods causes mappers to fail
Date Thu, 20 Dec 2012 21:19:22 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537382#comment-13537382

Harsh J commented on MAPREDUCE-4897:

A simple workaround until this is resolved may be to add a job.setMapper(…) call at the
end of job configuration, after MI but before submit, causing it to override MI's changes
and making things work again.
> Using both MultipleInputs.addInputPath methods causes mappers to fail
> ---------------------------------------------------------------------
>                 Key: MAPREDUCE-4897
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4897
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: gladmon@gmail.com
> The MultipleInputs class has two methods for adding inputs, one takes a mapper for the
provided input path. When using both, both the inputformat gets set to DelegatingInputFormat
class and the mapper class gets set to the DelegatingMapperClass.
> When running a map task, the delegating input sees theres no mapper for the provided
input path, and the next step is to get the mapper from the jobconf (which usually defaults
to the identity mapper). However, because I used both methods in the MultipleInputs class,
the mapper in the jobconf is set to the DelegatingMapper. Thus the delegating mapper creates
a delegating mapper and tries its map method. (this is how the framework gets the delegating
mapper to use a provided mapper). And the map method sees no mapper created yet, creates a
new delegating mapper, uses it, which see no mapper created yet, creates a delegating mapper,

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message