hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahil Varshney <shahilvarsh...@gmail.com>
Subject Re: map() function call related
Date Tue, 07 Apr 2015 18:15:25 GMT
thanks chris :)

On Tue, Apr 7, 2015 at 10:43 PM, Chris Nauroth <cnauroth@hortonworks.com>
wrote:

> Hello Shahil,
>
> In the current trunk codebase, the relevant files are
> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-co
> re/src/main/java/org/apache/hadoop/mapred/MapTask.java and
> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-co
> re/src/main/java/org/apache/hadoop/mapreduce/Mapper.java.  MapTask manages
> the execution of the mapper task, and eventually it calls Mapper#run,
> which then calls into the implementation of the map method.  BTW, you'll
> also see a corresponding ReduceTask.java and Reducer.java in the same
> directories if you need to look at those too.
>
> Input split calculation is performed by a subclass of InputFormat.
>
> http://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/mapreduce/InputF
> ormat.html
>
>
> I recommend looking at that.  You also can navigate down through those
> JavaDocs to identify subclasses of InputFormat, like FileInputFormat and
> TextInputFormat, which you can then find in the source code.
>
> I hope this helps.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
>
>
>
> On 4/7/15, 6:09 AM, "Shahil Varshney" <shahilvarshney@gmail.com> wrote:
>
> >Sir ,
> >i want to know that which class in hadoop (internal source class) is
> >responsible for calling map function for each key value pair(means calls
> >map() function).
> >
> > and which class actually done the input split job. i want to create my
> >own
> >class for input split so please tell me .
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message