hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Baldeschwieler <eri...@hortonworks.com>
Subject Re: Large feature development (YARN vs HDFS)
Date Mon, 03 Sep 2012 21:30:01 GMT

Referring back to Chris M.s thread, this YARN vs HDFS discussion sounds a lot like an umbrella
project issue to me.

On Sep 2, 2012, at 3:11 PM, Arun Murthy wrote:

> Eli,
> 
> On Sep 2, 2012, at 1:01 PM, Eli Collins <eli@cloudera.com> wrote:
> 
>> On Sat, Sep 1, 2012 at 12:47 PM, Arun C Murthy <acm@hortonworks.com> wrote:
>>> Todd,
>>> 
>>> On Sep 1, 2012, at 1:20 AM, Todd Lipcon wrote:
>>> 
>>>> I'd actually contend that YARN was merged too early. I have yet to see
>>>> anyone running YARN in production, and it's holding up the "Stable"
>>>> moniker for Hadoop 2.0 -- HDFS-wise we are already quite stable and
>>>> I'm seeing fewer issues in our customers running Hadoop HDFS 2
>>>> compared to Hadoop 1-derived code.
>>> 
>>> You know I respect you a ton, but I'm very saddened to see you perpetuate this
FUD on our public lists. I expected better, particularly when everyone is working towards
the same goals of advancing Hadoop-2. This sniping on other members doing work is, um, I'll
just stop here rather than regret later.
>> 2. HDFS is more mature than YARN. Not a surprise given that we all
>> agree YARN is alpha, and a much newer project than HDFS that hasn't
>> yet been deployed in production environments yet (to my knowledge).
> 
> Let's focus on the ground reality here.
> 
> Please read my (or Rajiv's) message again about YARN's current
> stability and how much it's baked, it's deployment plans to a very
> large cluster in a few *days*. Or, talk to the people developing,
> testing and supporting these customers and clusters.
> 
> I'll repeat - YARN has clearly baked much more than HDFS HA given
> the basic bugs (upgrade, edit logs corruption etc.) we've seen after
> being declared *done*; but then we just disagree since clearly I'm
> more conservative. Also, we need to be more conservative wrt HDFS -
> but then what would I know...
> 
> I'll admit it's hard to discuss with someone (or a collective) who
> just repeat themselves. Plus, I broke my own rule about email this
> weekend - so, I'll try harder.
> 
> Arun


Mime
View raw message