hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milind A Bhandarkar <mili...@yahoo-inc.com>
Subject Re: bringing the codebases back in line
Date Fri, 22 Oct 2010 04:29:30 GMT
> right.. the trunk is not for production use.  I wasn't suggesting that.

So, what are you suggesting ? That Yahoo distribution of Hadoop should *not* be the version
we run on our production clusters ?

> but the trunk is what will eventually become the next release.

> Then someone in yahoo will have to decide if they are going to move to
> rebase their production cluster to 0.21, or just continue back-porting what
> they need to the version they are running on their clusters.

Yes, that is what we do now. If there are committed patches in trunk that do not scale for
our needs, or break existing applications, or are deemed not worth the efforts needed to backport,
we do not include them in our deployments, and therefore do not include in Yahoo distribution.

> and if yahoo fixes a bug in their version, it would need to be
> forward-ported over to the current trunk. which will get harder and harder
> as the paths diverge.

Yes, indeed. So, care must be taken that paths do not diverge too much. I have seen some cases
where the bug fixes did not need to be forward ported, because that piece of code was completely
re-written in trunk.

> I'm sure you've seen it happen on other projects when a major branch lands
> on the trunk, and the amount of effort it takes to reconcile them.

Yes. And that results in delayed releases. An unexpected benefit for application developers
was that they could spend time adding features to their applications, rather than porting
same applications from release-to-release, and validating releases. So, it's not always bad.

- Milind

Milind Bhandarkar
(phone: 408-203-5213 W)

View raw message