hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: abandoning 22 - was: Content request for 0.20.205 Sustaining Release
Date Fri, 09 Sep 2011 18:26:58 GMT

On Sep 9, 2011, at 10:41 AM, Rottinghuis, Joep wrote:

No one is going to block you from doing any work you want. 

All that is required is to have the work in trunk and subsequent branches i.e 0.23 (as applicable)

The problem is that 0.22 hasn't seen major movement for almost a year since it was branched
and there is no incentive for lots of people to contribute - plus there is MRv2 which is completely
different beast (see the discussion that Eli pointed to).

None of this is to say you shouldn't contribute or use 0.22, you move the project as you wish
by your contributions.

>> MR1 is being maintained on 20x. In fact 20x is the only MR1 code that supports security
and disk failure handling. The MR1 code in 22 is a regression in some significant aspects
(features, performance, bugs) from the latest stable MR1 (204).
> ...
> Eli, aside from the disk failure handling which is a new feature in 205 (not present
in earlier 20x releases), could you please elaborate on which other significant aspects 22
would regress from 20x? 

I've talked to Konstantin about this.

There is tonnes of performance work missing, including scaling work on JobTracker, CapacityScheduler
etc. There is work to add a ton of limits (counters, tasks, etc. etc.). Then there is operability
work such as JobHistory, handling logs etc. The last time I benchmarked 22 v/s 20.xxx there
was >5x difference and that was more than a year ago. Arguably some of the operability
work won't matter for small clusters, but you are welcome to make your own decisions.

It's unfortunate we have landed here, but 22 branched almost a year ago and hence none of
this work was ported there since a branch implies critical bugs was supposed to go in. That
plus the problems with scaling MR1 which led to investment in MR1 is where we are. As a result,
there is no enthusiasm to contribute to MR1 from vast majority of devs given that we've decided
we won't support it. 

>> My understanding of the way Apache operates is that you can't do things like "declare
blocks on upgrade paths". People can try to release updates to 21 or 22 (or some new tree).
Ie the decisions are made implicitly by where people invest cycles.
> If a group of committers say that they'll commit to trunk, to 0.23 and to 0.20x, but
not to 0.22, then in effect that is like to "declare a block on upgrade path" isn't it? The
more such commits that go in to other branches, but not the ones in between essentially is
a declaration of a block, because of the very regression argument you make.
> I agree that nobody can be made to contribute to something they don't want, but does
that result into a split?
> In other words, if a significant bug fix or feature goes into trunk and into 22, can
developers then simply say: "I'm not interested to put this into 23, you do this yourself
if you want?". Will that be tolerated or vetoed?

Again, no one is going to veto anything. As Eli said decisions are make by people's code.


View raw message