hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "gladmon@gmail.com (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4897) Using both MultipleInputs.addInputPath methods causes mappers to fail
Date Thu, 20 Dec 2012 21:01:13 GMT
gladmon@gmail.com created MAPREDUCE-4897:
--------------------------------------------

             Summary: Using both MultipleInputs.addInputPath methods causes mappers to fail
                 Key: MAPREDUCE-4897
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4897
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 0.20.2
            Reporter: gladmon@gmail.com


The MultipleInputs class has two methods for adding inputs, one takes a mapper for the provided
input path. When using both, both the inputformat gets set to DelegatingInputFormat class
and the mapper class gets set to the DelegatingMapperClass.

When running a map task, the delegating input sees theres no mapper for the provided input
path, and the next step is to get the mapper from the jobconf (which usually defaults to the
identity mapper). However, because I used both methods in the MultipleInputs class, the mapper
in the jobconf is set to the DelegatingMapper. Thus the delegating mapper creates a delegating
mapper and tries its map method. (this is how the framework gets the delegating mapper to
use a provided mapper). And the map method sees no mapper created yet, creates a new delegating
mapper, uses it, which see no mapper created yet, creates a delegating mapper, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message