hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Update on hadoop-0.23
Date Fri, 30 Sep 2011 17:54:11 GMT

On Sep 30, 2011, at 3:17 AM, Steve Loughran wrote:
> 
> 3. There's 0.22 which is going to combine the API of 0.21 with the fixes of 0.20.20x
*and* will be the last release of the MR1.0 engine. For that last reason, I think there's
value in pushing it out, though it's going to take time, and there's a risk of it adding another
branch to be maintained for an indeterminate period.

I'm all for people working on what they are passionate about, so this isn't to say one shouldn't
spend time on 0.22.

But, for clarity's sake, as I've done multiple times on both the list and in person to Konstantin
etc., I'll point out (again) that 0.22 will need multiple man-years of development to achieve
parity with 0.20.2xx just in terms of bug-fixes and performance. Then there is security, multi-tenancy
etc. which regress significantly vis-a-vis 0.20.2xx. Then there is scaling etc.

0.23 is already past all of these hurdles and very close to meeting, if not beating 0.20.2xx
in performance. It already beats 0.20.2xx in lots of dimensions (improved shuffle with zero-copy
etc.).

So, unless folks plan to invest this gargantuan time, please do not say that 0.21 has fixes
from 0.20.2xx. That's all I ask. Thus, 0.20.2xx may well be the last _viable_ release of MR1
engine.

Arun


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message