hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Kendall <mkend...@justin.tv>
Subject Re: only one reduce task?
Date Fri, 04 Dec 2009 00:36:40 GMT
yup, only one task...

i should have mentioned that i'm using hadoop streaming.  do i have
access to jobconf* if i write my tasks in python?

On Thu, Dec 3, 2009 at 4:32 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
> Mike,
>
> Do you mean you only have one reducer task for a Job ?
>
> You can increase the number of reducer task for one Job by setting
>
> JobConf.setNumReduceTasks(n)
>
>
> Jeff Zhang
>
>
> On Thu, Dec 3, 2009 at 2:58 PM, Mike Kendall <mkendall@justin.tv> wrote:
>
>> i can't seem to get my cluster to run more than one reduce task...  my
>> mapred-site.xml looks like this:
>>
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>>
>> <!-- Put site-specific property overrides in this file. -->
>> <configuration>
>> <property>
>>  <name>mapred.job.tracker</name>
>>  <value>master:9001</value>
>> </property>
>> <property>
>>  <name>mapred.tasktracker.map.tasks.maximum</name>
>>  <value>5</value>
>> </property>
>>
>> <property>
>>  <name>mapred.tasktracker.reduce.tasks.maximum</name>
>>  <value>5</value>
>> </property>
>> <property>
>>  <name>mapred.map.tasks</name>
>>  <value>40</value>
>> </property>
>> <property>
>>  <name>mapred.reduce.tasks</name>
>>  <value>8</value>
>> </property>
>> <property>
>>  <name>mapred.jobtracker.taskScheduler</name>
>>  <value>org.apache.hadoop.mapred.FairScheduler</value>
>> </property>
>> <property>
>>  <name>mapred.fairscheduler.allocation.file</name>
>>  <value>/usr/local/hadoop/conf/fairshare-pools.xml</value>
>> </property>
>> </configuration>
>>
>> any ideas?  thanks.
>>
>

Mime
View raw message