hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1743) conf.get("map.input.file") returns null when using MultipleInputs in Hadoop 0.20
Date Fri, 30 Apr 2010 00:10:54 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862485#action_12862485
] 

Tom White commented on MAPREDUCE-1743:
--------------------------------------

> One solution is to let TaggedInputSplit extend FileSplit.

Except that MultipleInputs works with any InputFormat, not just FileInputFormat. Another way
of doing this would be to invert the logic so that the InputSplit updates properties on Configuration
(this would need a new method on InputSplit).

> conf.get("map.input.file") returns null when using MultipleInputs in Hadoop 0.20
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1743
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1743
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: Yuanyuan Tian
>
> There is a problem in getting the input file name in the mapper when uisng MultipleInputs
in Hadoop 0.20. I need to use MultipleInputs to support different formats for my inputs to
the my MapReduce job. And inside each mapper, I also need to know the exact input file that
the mapper is processing. However, conf.get("map.input.file") returns null. Can anybody help
me solve this problem? Thanks in advance.
> public class Test extends Configured implements Tool{
> 	static class InnerMapper extends MapReduceBase implements Mapper<Writable, Writable,
NullWritable, Text>
> 	{
> 		................
> 		................
> 		public void configure(JobConf conf)
> 		{	
> 			String inputName=conf.get("map.input.file"));
> 			.......................................
> 		}
> 		
> 	}
> 	
> 	public int run(String[] arg0) throws Exception {
> 		JonConf job;
> 		job = new JobConf(Test.class);
> 		...........................................
> 		
> 		MultipleInputs.addInputPath(conf, new Path("A"), TextInputFormat.class);
> 		MultipleInputs.addInputPath(conf, new Path("B"), SequenceFileFormat.class);
> 		...........................................
> 	}
> }

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message