hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: Build/test infrastructure
Date Sat, 26 Feb 2011 20:43:55 GMT
We should be very careful about the approach that we chosen for build/packaging.  The current
state of hadoop is coupled together due to lack of standardized RPC format.  Once this issue
is cleared, the community will want to split hdfs and m/r into separated projects at some
point.  It may be better to ensure project is modularized, and work from the same svn repository.
 Maven is great for doing this, and most of the build and scripts can be defined in pom.xml.
 Deployment/test server configuration can be pass in from hudson.  We should ensure that build
and deployment script do not further couple the project.


On 2/26/11 11:14 AM, "Konstantin Boudnik" <cos@apache.org> wrote:

On Fri, Feb 25, 2011 at 23:47, Nigel Daley <ndaley@mac.com> wrote:
> +1.
> Once HADOOP-7106 is committed, I'd like to propose we create a directory at the same
level of common/hdfs/mapreduce to hold build (and deploy) type scripts and files.  These would
then get branches/tagged with the rest of the release.

That makes sense, although I don't see changes of the host
configurations to happen very often.


> Nige
> On Feb 25, 2011, at 7:55 PM, Konstantin Boudnik wrote:
>> Looking at re-occurring build/test-patch problems on hadoop? build machines I
>> thought of a way to make them:
>>  a) all the same (configuration, installed software wise)
>>  b) have an effortless system to run upgrades/updates on all of them in a
>>  controlled fashion.
>> I would suggest to create Puppet configs (the exact content to be defined)
>> which we'll be checked in SCM (e.g. SVN), Whenever a build host's software
>> is needed to be restored/updated a simple run of Puppet across the machines
>> or change in config and run of Puppet will do the magic for us.
>> If there are no objections from the community I can put together some
>> Puppet recipes which might be evolved as we go.
>> --
>> Take care,
>>       Cos
>> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>> After all, it is only the mediocre who are always at their best.
>>                Jean Giraudoux

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message