hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Gao <steve....@yahoo.com>
Subject Help: How to change number of mappers in Hadoop streaming?
Date Thu, 16 Oct 2008 19:55:15 GMT

Would anybody help me?
Can I use 
-jobconf mapred.map.task=50 in streaming command to change the job's number of mappers?

I don't have a hadoop at hand and can not verify it. Thanks for your help.

--- On Wed, 10/15/08, Steve Gao <steve.gao@yahoo.com> wrote:
From: Steve Gao <steve.gao@yahoo.com>
Subject: How to change number of mappers in Hadoop streaming?
To: core-user@hadoop.apache.org
Cc: core-dev@hadoop.apache.org
Date: Wednesday, October 15, 2008, 7:25 PM

Is there a way to change number of mappers in Hadoop streaming command line?
I know I can change hadoop-default.xml:

<property>
  <name>mapred.map.tasks</name>
  <value>10</value>
  <description>The default number of map tasks per job.  Typically set
  to a prime several times greater than number of available hosts.
  Ignored when mapred.job.tracker is "local".
  </description>
</property>

But that's for all jobs. What if I just want each job has different
NUM_OF_Mappers themselves? Thanks




      


      
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message