hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amit handa <amha...@gmail.com>
Subject org.apache.hadoop.mapreduce.Mapper setup method equivalent in streaming or pipes
Date Tue, 02 Jun 2009 04:00:30 GMT

I want to do some initial one time set up before each of my map task starts.
Is there some method which is similar to
org.apache.hadoop.mapreduce.Mapper -setup method, in hadoop pipes or
streaming ?

Another newbie question - Does hadoop keep the map tasks alive for a given
job e.g if i set max map tasks to 4 and  if i call setup method , can i
assume that it won't be called more than 4 times (once for each map task)?
I am assuming the task runs normally (no task hangs or crashes)


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message