hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: bringing the codebases back in line
Date Fri, 22 Oct 2010 11:45:28 GMT
On 22/10/10 01:42, Milind A Bhandarkar wrote:
>> but the other question I have which hopefully you guys can answer is does
>> the yahoo distribution have ALL the patches from the trunk on it? because if
>> it doesn't I think that is problematic as well for other reasons.
> What are these "other" reasons ?
> yahoo distribution runs on our production clusters, and I do not see why any production
cluster should run code from trunk.

Transient virtual clusters where the FS only exists for a couple of 
hours can do this, but big live physical ones shouldn't, too much data 
at risk. So it depends on your view of "production". If it is "someone 
wants a cluster for 3h to analyse their nightly logs", then you can get 
away with it -it's the best testing you can do.

One problem I hit doing this is that if you do upgrade every week, if 
the app starts failing, you can waste a lot of time trying to decide 
whether its the app code that's changed or trunk, and then if its trunk, 
whether that's a regression or a change that's caused a bug in the app 
code to surface. To be really strict, you should A/B test your virtual 
clusters on the stable and trunk versions, see which finishes faster 
-and whether there are any differences in the output.

that would be very slick for testing indeed.

View raw message