Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: <2E938682-2109-4487-96BF-35C4497D04ED@hortonworks.com>
References: <2E938682-2109-4487-96BF-35C4497D04ED@hortonworks.com>
Date: Mon, 23 Nov 2015 01:12:25 +0900
Message-ID: 
 <CAAD07OJf848Y-5D0oD1SP1AThw9QNNX4UWHxDnsJVoO+VWaRfA@mail.gmail.com>
Subject: Re: Jenkins stability and patching
From: Tsuyoshi Ozawa <ozawa@apache.org>
To: "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>
Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>,
	"common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thank you for starting discussion, Steve. It sounds good to me.  I'll
check the test failures.

- Tsuyoshi

On Sun, Nov 22, 2015 at 9:21 PM, Steve Loughran <stevel@hortonworks.com> wr=
ote:
>
> Jenkins is pretty much dead in the water these days; a test run that work=
s is a rare miracle rather than the default state. Which also means most pa=
tches are being +1'd in even though patches are failing, with comments like=
 "the test failures are probably unrelated"
>
>
> I think everyone has to be grateful that I'm not volunteering to be relea=
se manager for 2.8, as if I were i'd have already imposed a block on any pa=
tches going in until jenkins was stable. That is: nothing but test fixes wo=
uld go in.
>
> as it is, at least for the next couple of weeks, I'm going to experiment =
with reverting patches which break the build. Usually those breakages are b=
eing fixed, eventually, with followup patches. With a "patches which break =
the build get reverted" policy, whoever submitted that first patch gets to =
write the fix *and test it again*. This should encourage people to be more =
rigorous first time round.
>
>
>   1.  Yes, I'm going to have to be ruthless and do this for myself too. O=
r others can. I'm not doing much (any?) core hadoop coding right now, so mo=
re isolated.
>   2.  No, I don't plan to show favouritism: break the build and it gets r=
olled back.
>   3.  We can review this in a week or two  to see how it goes. And someon=
e else can volunteer to keep jenkins happy.
>   4.  I'll get a smaller fix for HDFS-9263 in.
>   5.  I've also started running slider 0.90-SNAPSHOT test runs with Hadoo=
p 2.8.0-SNAPSHOT, so I'm being the first to find problems beyond jenkins. S=
o far HADOOP-12050 is the first blocker. It went in in August, which shows =
we aren't doing enough cross-version testing beyond just Jenkins. That brea=
kage (HADOOP-12587) is stopping my test code working against secure cluster=
s =E2=80=94if I was being really harsh I'd have reverted that too, but's be=
en in long enough I think a fix is probably the best solution.
>   6.  Finally: everyone should feel free to fix tests. Don't be shy now!
>
> Giving this is a US vacation week, it should be a quieter week for breaka=
ges.
>
> Sorry =E2=80=94but if we can't even get Jenkins stable, then what hope do=
 we have for a 2.8 release working?
>
> -Steve
>
>