From general-return-3127-apmail-hadoop-general-archive=hadoop.apache.org@hadoop.apache.org Sat Feb 26 22:19:30 2011 Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 72261 invoked from network); 26 Feb 2011 22:19:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 26 Feb 2011 22:19:29 -0000 Received: (qmail 47516 invoked by uid 500); 26 Feb 2011 22:19:28 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 47463 invoked by uid 500); 26 Feb 2011 22:19:28 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 47455 invoked by uid 99); 26 Feb 2011 22:19:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 26 Feb 2011 22:19:27 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.125.82.176] (HELO mail-wy0-f176.google.com) (74.125.82.176) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 26 Feb 2011 22:19:22 +0000 Received: by wya21 with SMTP id 21so3375546wya.35 for ; Sat, 26 Feb 2011 14:19:01 -0800 (PST) Received: by 10.216.17.201 with SMTP id j51mr3441644wej.44.1298758741056; Sat, 26 Feb 2011 14:19:01 -0800 (PST) MIME-Version: 1.0 Sender: cos@boudnik.org Received: by 10.216.180.70 with HTTP; Sat, 26 Feb 2011 14:18:41 -0800 (PST) In-Reply-To: References: From: Konstantin Boudnik Date: Sat, 26 Feb 2011 14:18:41 -0800 X-Google-Sender-Auth: 9HdhXLMp0soAsMApVoaNT1P07mo Message-ID: Subject: Re: Build/test infrastructure To: general@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable This discussion isn't about build of the product nor about packaging of it. We are discussing patch validation and snapshot build infrastructure. On Sat, Feb 26, 2011 at 12:43, Eric Yang wrote: > We should be very careful about the approach that we chosen for build/pac= kaging. =A0The current state of hadoop is coupled together due to lack of s= tandardized RPC format. =A0Once this issue is cleared, the community will w= ant to split hdfs and m/r into separated projects at some point. =A0It may = be better to ensure project is modularized, and work from the same svn repo= sitory. =A0Maven is great for doing this, and most of the build and scripts= can be defined in pom.xml. =A0Deployment/test server configuration can be = pass in from hudson. =A0We should ensure that build and deployment script d= o not further couple the project. > > Regards, > Eric > > On 2/26/11 11:14 AM, "Konstantin Boudnik" wrote: > > On Fri, Feb 25, 2011 at 23:47, Nigel Daley wrote: >> +1. >> >> Once HADOOP-7106 is committed, I'd like to propose we create a directory= at the same level of common/hdfs/mapreduce to hold build (and deploy) type= scripts and files. =A0These would then get branches/tagged with the rest o= f the release. > > That makes sense, although I don't see changes of the host > configurations to happen very often. > > Cos > >> Nige >> >> On Feb 25, 2011, at 7:55 PM, Konstantin Boudnik wrote: >> >>> Looking at re-occurring build/test-patch problems on hadoop? build mach= ines I >>> thought of a way to make them: >>> =A0a) all the same (configuration, installed software wise) >>> =A0b) have an effortless system to run upgrades/updates on all of them = in a >>> =A0controlled fashion. >>> >>> I would suggest to create Puppet configs (the exact content to be defin= ed) >>> which we'll be checked in SCM (e.g. SVN), Whenever a build host's softw= are >>> is needed to be restored/updated a simple run of Puppet across the mach= ines >>> or change in config and run of Puppet will do the magic for us. >>> >>> If there are no objections from the community I can put together some >>> Puppet recipes which might be evolved as we go. >>> >>> -- >>> Take care, >>> =A0 =A0 =A0 Cos >>> 2CAC 8312 4870 D885 8616 =A06115 220F 6980 1F27 E622 >>> >>> After all, it is only the mediocre who are always at their best. >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Jean Giraudoux >> >> > >