hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samaneh Shokuhi <samaneh.shok...@gmail.com>
Subject question :sorting in MapTask and straggler mapper
Date Sun, 18 Nov 2012 14:04:15 GMT
i have two questions :
1) As i understood ,before shuffling the data to reducer, kind of sorting
is done on Map phase (MapTask.java).
I want to do some experiments by disabling this sorting .
I think this sorter [ private final IndexedSorter sorter; ]  is responsible
to do sort in  sortAndSpill  method .so if i remove this part, the sorting
will be disabled in map phase.  Am i right or it is more complicated than i
think ?

2) My second question is ,i need to detect the straggler map tasks
specifically the last one.As i know job tracker is aware of all map tasks
,so is there any way to know the current map task is the last(straggler)
one  ?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message