hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Is it pissible get a number of mapper tasks?
Date Sat, 04 Dec 2010 01:59:56 GMT

(Answers may be 0.20 specific)

On Sat, Dec 4, 2010 at 6:41 AM, Jason <urgisb@gmail.com> wrote:
> In my mapper code I need to know the total number of mappers which is the same as number
of input splits.
> (I need it for unique int Id generation)

mapred.map.tasks is set for every job before launch and is the total
number of maps that are going to run for a successful result.

There are more identifiers, however:
mapred.task.id is the task's id (contains various info, map/reduce).
mapred.attempt.id is the task's _attempt_ id (basically task id, with
attempt information, map/reduce).
mapred.task.partition is the unique partition id per task (perhaps you
can utilize this integer itself, applies to map/reduce)

(Some of these vars have methods of setter/getter in JobConf itself)

> Basically Im looking for an analog of context.getNumReduceTasks() but can't find it.
> Thanks

Harsh J

View raw message