Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
From: Arun C Murthy <acm@hortonworks.com>
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: multipart/alternative; boundary=Apple-Mail-681--955715767
Subject: Re: Large feature development (YARN vs HDFS)
Date: Mon, 3 Sep 2012 17:31:11 -0700
In-Reply-To: <D63E4A37-9118-4A0D-A7D1-65A1F66977D4@hortonworks.com>
To: general@hadoop.apache.org
References: 
 <CADadbPE2LuOCbp4aUr0XiKsSQ2iPZ_D6qjc8MgphyvZ3YOo9pw@mail.gmail.com>
 <CADY20s6NuNckKD0JhQn72h4gTLaz-SC9w+9swfEa7RU7xi_RRg@mail.gmail.com>
 <751827D1-0353-45E3-B5AA-14E0556CDCEE@hortonworks.com>
 <CAPn_vTu0ihKp+sdiqh=nVNZT5x9HVfcCs1zECLaDvXNfQw0KKw@mail.gmail.com>
 <-7917683134968972508@unknownmsgid>
 <D63E4A37-9118-4A0D-A7D1-65A1F66977D4@hortonworks.com>
Message-Id: <F9E0BD64-B943-46AF-A1E3-9491F02F59A0@hortonworks.com>

--Apple-Mail-681--955715767
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Agreed... it does seem like a case of 'my wife is prettier'.

Maybe I'm oversensitive and it may even be understandable given how much =
of my waking time I've devoted to YARN over the last 30 months; but I do =
apologize for indulging in the behavior I accused others of. A good =
night's sleep does help in clearing mists. IAC, the point I was trying =
to quantify is simple - current state of YARN is far better than was =
being characterized here.

We should get back to discussing 'large-feature development' - thanks =
for starting that discussion Steve.

Arun

On Sep 3, 2012, at 2:30 PM, Eric Baldeschwieler wrote:

>=20
> Referring back to Chris M.s thread, this YARN vs HDFS discussion =
sounds a lot like an umbrella project issue to me.
>=20
> On Sep 2, 2012, at 3:11 PM, Arun Murthy wrote:
>=20
>> Eli,
>>=20
>> On Sep 2, 2012, at 1:01 PM, Eli Collins <eli@cloudera.com> wrote:
>>=20
>>> On Sat, Sep 1, 2012 at 12:47 PM, Arun C Murthy <acm@hortonworks.com> =
wrote:
>>>> Todd,
>>>>=20
>>>> On Sep 1, 2012, at 1:20 AM, Todd Lipcon wrote:
>>>>=20
>>>>> I'd actually contend that YARN was merged too early. I have yet to =
see
>>>>> anyone running YARN in production, and it's holding up the =
"Stable"
>>>>> moniker for Hadoop 2.0 -- HDFS-wise we are already quite stable =
and
>>>>> I'm seeing fewer issues in our customers running Hadoop HDFS 2
>>>>> compared to Hadoop 1-derived code.
>>>>=20
>>>> You know I respect you a ton, but I'm very saddened to see you =
perpetuate this FUD on our public lists. I expected better, particularly =
when everyone is working towards the same goals of advancing Hadoop-2. =
This sniping on other members doing work is, um, I'll just stop here =
rather than regret later.
>>> 2. HDFS is more mature than YARN. Not a surprise given that we all
>>> agree YARN is alpha, and a much newer project than HDFS that hasn't
>>> yet been deployed in production environments yet (to my knowledge).
>>=20
>> Let's focus on the ground reality here.
>>=20
>> Please read my (or Rajiv's) message again about YARN's current
>> stability and how much it's baked, it's deployment plans to a very
>> large cluster in a few *days*. Or, talk to the people developing,
>> testing and supporting these customers and clusters.
>>=20
>> I'll repeat - YARN has clearly baked much more than HDFS HA given
>> the basic bugs (upgrade, edit logs corruption etc.) we've seen after
>> being declared *done*; but then we just disagree since clearly I'm
>> more conservative. Also, we need to be more conservative wrt HDFS -
>> but then what would I know...
>>=20
>> I'll admit it's hard to discuss with someone (or a collective) who
>> just repeat themselves. Plus, I broke my own rule about email this
>> weekend - so, I'll try harder.
>>=20
>> Arun
>=20

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/


--Apple-Mail-681--955715767--