Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:user-agent:date:subject:from:to:message-id:
	thread-topic:thread-index:in-reply-to:mime-version:content-type:
	content-transfer-encoding:x-originalarrivaltime;
	b=G2mMLpkodsIq08JrQFQ/dqwI+MdR5j+GdXmwr5jqf/qV1XBh1G2ZVdec4cUXdynU
User-Agent: Microsoft-Entourage/12.17.0.090302
Date: Thu, 14 May 2009 11:34:02 -0700
Subject: Re: another rc4 issue?
From: Mac Yang <macyang@yahoo-inc.com>
To: <chukwa-dev@hadoop.apache.org>
Message-ID: <C631B22A.12567%macyang@yahoo-inc.com>
Thread-Topic: another rc4 issue?
Thread-Index: AcnUwo2Hww3BZKHqckihIjxIExX97A==
In-Reply-To: <39b0afc00905140933g7f47d247k4b1b956cbda73f05@mail.gmail.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable


Unfortunately that override / merging infrastructure is not completely up
yet. Once that's done then we will be able to handle this type of situation=
s
more cleanly. And I do agree that this needs to get addressed before we mak=
e
the external release.


On 5/14/09 9:33 AM, "Ariel Rabkin" <asrabkin@gmail.com> wrote:

> I just opened CHUKWA-239 for this.  It should be a near-trivial patch.
>=20
> Am I right in assuming that Yahoo! has internal corporate settings
> which will override chukwa-demux-conf anyway?
>=20
> --Ari
>=20
> On Thu, May 14, 2009 at 8:04 AM, Cheng Zhang <chzhang@yahoo-inc.com> wrot=
e:
>> +1. As an open source project, the default settings should be workable.
>>=20
>>=20
>>=20
>>> -----Original Message-----
>>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>> Sent: Wednesday, May 13, 2009 11:29 PM
>>> To: chukwa-dev@hadoop.apache.org
>>> Subject: another rc4 issue?
>>>=20
>>> Also, the demux mapred jobs all fail almost immediately with an
>>> out-of-memory. =A0The sort buffer sizes in chukwa-demux-conf are
>>> incompatible with the Hadoop defaults.
>>>=20
>>> I think we should dial down the default sizes in chukwa-demux-conf.
>>> Outside Yahoo!, much smaller buffers should still be OK, since nobody
>>> else will be running Chukwa at large scale for a while.
>>>=20
>>> --Ari
>>>=20
>>> --
>>> Ari Rabkin asrabkin@gmail.com
>>> UC Berkeley Computer Science Department
>>=20
>=20
>=20