Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of mapred.learn@gmail.com
 designates 209.85.213.48 as permitted sender)
References: 
 <CAFxxKifd_6eUCtBs15=rdD0mKWtC_cHAoLYcH8_m4mgjLhbsJw@mail.gmail.com>
 <173778479550729024@unknownmsgid>
 <13F54E3A-149F-4FED-91EF-6DA1ED1A8F40@gmail.com>
 <08D2EC68-F4F1-4E75-8BA4-5E281ACC14EA@cloudera.com>
In-Reply-To: <08D2EC68-F4F1-4E75-8BA4-5E281ACC14EA@cloudera.com>
Mime-Version: 1.0 (iPhone Mail 8F190)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii
Message-Id: <8C55A5B7-5988-4A6F-9CAD-82107F808498@gmail.com>
Cc: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
From: Mapred Learn <mapred.learn@gmail.com>
Subject: Re: Streaming jar creates only 1 reducer
Date: Fri, 21 Oct 2011 22:22:28 -0700
To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>

Thanks Harsh !
This is exactly what I thought !

And don't know what you mean by cross-post ? I just posted to mapred and HDFS=
 mailing lists ? What's your point about cross-pointing ??

Sent from my iPhone

On Oct 21, 2011, at 8:57 PM, Harsh J <harsh@cloudera.com> wrote:

> Mapred,
>=20
> You need to pass -Dmapred.reduce.tasks=3DN along. Reducers are a per-job c=
onfigurable number, unlike mappers whose numbers can be determined based on i=
nputs.
>=20
> P.s. Please do not cross post questions to multiple lists.
>=20
> On 22-Oct-2011, at 4:05 AM, Mapred Learn wrote:
>=20
>> Do you know what parameters from conf files ?
>>=20
>> Thanks,
>>=20
>> Sent from my iPhone
>>=20
>> On Oct 21, 2011, at 3:32 PM, Nick Jones <darellik@gmail.com> wrote:
>>=20
>>> FWIW, I usually specify the number of reducers in both streaming and
>>> against the Java API. The "default" is what's read from your config
>>> files on the submitting node.
>>>=20
>>> Nick Jones
>>>=20
>>> On Oct 21, 2011, at 5:00 PM, Mapred Learn <mapred.learn@gmail.com> wrote=
:
>>>=20
>>>> Hi,
>>>> Does streaming jar create 1 reducer by default ? We have reduce tasks p=
er task tracker configured to be more than 1 but my job has about 150 mapper=
s and only 1 reducer:
>>>>=20
>>>> reducer.py basically just reads the line and prints it.
>>>>=20
>>>> Why doesn't streaming.jar invokes multiple reducers for this case ?
>>>>=20
>>>> Thanks,
>>>> -JJ
>>>>=20
>>>>=20
>=20