Return-Path: 
 <common-user-return-19236-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: (qmail 42513 invoked from network); 4 Dec 2009 06:40:22 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 4 Dec 2009 06:40:22 -0000
Received: (qmail 28798 invoked by uid 500); 4 Dec 2009 06:40:20 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 28702 invoked by uid 500); 4 Dec 2009 06:40:19 -0000
Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-user@hadoop.apache.org>
List-Id: <common-user.hadoop.apache.org>
Reply-To: common-user@hadoop.apache.org
Delivered-To: mailing list common-user@hadoop.apache.org
Received: (qmail 28692 invoked by uid 99); 4 Dec 2009 06:40:19 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Dec 2009 06:40:19 +0000
X-ASF-Spam-Status: No, hits=-2.5 required=5.0
	tests=AWL,BAYES_00,HTML_MESSAGE,NO_RDNS_DOTCOM_HELO
X-Spam-Check-By: apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Received: from [69.147.107.21] (HELO mrout2-b.corp.re1.yahoo.com)
 (69.147.107.21)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Dec 2009 06:40:17 +0000
Received: from EGL-EX07CAS02.ds.corp.yahoo.com
 (egl-ex07cas02.eglbp.corp.yahoo.com [203.83.248.209])
	by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id
 nB46d4Ea036052
	for <common-user@hadoop.apache.org>; Thu, 3 Dec 2009 22:39:05 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:from:to:date:subject:thread-topic:thread-index:
	message-id:in-reply-to:accept-language:content-language:
	x-ms-has-attach:x-ms-tnef-correlator:acceptlanguage:content-type:mime-version;
	b=LMxmQUz/7mDqHkMDrrYB9yS+M4MtMJFwUNXFFlBMKhfPxFebKsBXZbR1ZE6rqab4
Received: from EGL-EX07VS01.ds.corp.yahoo.com ([203.83.248.206]) by
 EGL-EX07CAS02.ds.corp.yahoo.com ([203.83.248.216]) with mapi; Fri, 4 Dec 2009
 12:09:03 +0530
From: Amogh Vasekar <amogh@yahoo-inc.com>
To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org>
Date: Fri, 4 Dec 2009 12:09:02 +0530
Subject: Re: only one reduce task?
Thread-Topic: only one reduce task?
Thread-Index: Acp0efmWuV1jS9bfSjursicbdi4T/gAMn3RH
Message-ID: <C73EAC5E.4A36%amogh@yahoo-inc.com>
In-Reply-To: <2d9b8f430912031636x221a14bbw2c6e7664853309d3@mail.gmail.com>
Accept-Language: en-US
Content-Language: en
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_C73EAC5E4A36amoghyahooinccom_"
MIME-Version: 1.0

--_000_C73EAC5E4A36amoghyahooinccom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,
If you want to access certain jobconf parameters in your streaming script, =
streaming provides this by setting localized jobconf parameters as system e=
nvironment variables, with the "." in parameters replaced by "_" .
To set jobconf parameters for streaming jobs, you can use -D <param.name>=
=3D<value>

Thanks,
Amogh

On 12/4/09 6:06 AM, "Mike Kendall" <mkendall@justin.tv> wrote:

yup, only one task...

i should have mentioned that i'm using hadoop streaming.  do i have
access to jobconf* if i write my tasks in python?

On Thu, Dec 3, 2009 at 4:32 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
> Mike,
>
> Do you mean you only have one reducer task for a Job ?
>
> You can increase the number of reducer task for one Job by setting
>
> JobConf.setNumReduceTasks(n)
>
>
> Jeff Zhang
>
>
> On Thu, Dec 3, 2009 at 2:58 PM, Mike Kendall <mkendall@justin.tv> wrote:
>
>> i can't seem to get my cluster to run more than one reduce task...  my
>> mapred-site.xml looks like this:
>>
>> <?xml version=3D"1.0"?>
>> <?xml-stylesheet type=3D"text/xsl" href=3D"configuration.xsl"?>
>>
>> <!-- Put site-specific property overrides in this file. -->
>> <configuration>
>> <property>
>>  <name>mapred.job.tracker</name>
>>  <value>master:9001</value>
>> </property>
>> <property>
>>  <name>mapred.tasktracker.map.tasks.maximum</name>
>>  <value>5</value>
>> </property>
>>
>> <property>
>>  <name>mapred.tasktracker.reduce.tasks.maximum</name>
>>  <value>5</value>
>> </property>
>> <property>
>>  <name>mapred.map.tasks</name>
>>  <value>40</value>
>> </property>
>> <property>
>>  <name>mapred.reduce.tasks</name>
>>  <value>8</value>
>> </property>
>> <property>
>>  <name>mapred.jobtracker.taskScheduler</name>
>>  <value>org.apache.hadoop.mapred.FairScheduler</value>
>> </property>
>> <property>
>>  <name>mapred.fairscheduler.allocation.file</name>
>>  <value>/usr/local/hadoop/conf/fairshare-pools.xml</value>
>> </property>
>> </configuration>
>>
>> any ideas?  thanks.
>>
>


--_000_C73EAC5E4A36amoghyahooinccom_--