hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: map() function call related
Date Tue, 07 Apr 2015 17:13:49 GMT
Hello Shahil,

In the current trunk codebase, the relevant files are
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-co
re/src/main/java/org/apache/hadoop/mapred/MapTask.java and
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-co
re/src/main/java/org/apache/hadoop/mapreduce/Mapper.java.  MapTask manages
the execution of the mapper task, and eventually it calls Mapper#run,
which then calls into the implementation of the map method.  BTW, you'll
also see a corresponding ReduceTask.java and Reducer.java in the same
directories if you need to look at those too.

Input split calculation is performed by a subclass of InputFormat.

http://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/mapreduce/InputF
ormat.html


I recommend looking at that.  You also can navigate down through those
JavaDocs to identify subclasses of InputFormat, like FileInputFormat and
TextInputFormat, which you can then find in the source code.

I hope this helps.

Chris Nauroth
Hortonworks
http://hortonworks.com/






On 4/7/15, 6:09 AM, "Shahil Varshney" <shahilvarshney@gmail.com> wrote:

>Sir ,
>i want to know that which class in hadoop (internal source class) is
>responsible for calling map function for each key value pair(means calls
>map() function).
>
> and which class actually done the input split job. i want to create my
>own
>class for input split so please tell me .


Mime
View raw message