hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Pass the TaskId from map to Reduce
Date Sun, 14 Feb 2010 00:02:32 GMT
The number of map tasks is normally different from the number of reduce tasks.
Coding as you planned would limit the flexibility of hadoop.

On Saturday, February 13, 2010, ANKITBHATNAGAR <abhatnagar@vantage.com> wrote:
> Hi,
> I was working on a scenario where in I am generating a file in close()
> function of my Map implementation.
> Since Map execution are worked concurrently, this file is overwritten.
> I was wondering how to name this file uniquely per map execution basic and
> then read in configure() function of reduce.
> I could give a task id as name of the file but dont know how will I read the
> same file in configure() as the task id would have changed
> Ankit
> --
> View this message in context: http://old.nabble.com/Pass-the-TaskId-from-map-to-Reduce-tp27575531p27575531.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.

View raw message