Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@crunch.apache.org
Received-SPF: pass (nike.apache.org: domain of stepinto@live.com designates
 65.55.111.108 as permitted sender)
Message-ID: <BLU0-SMTP119A2075AC1852C91AEA886ADF20@phx.gbl>
MIME-Version: 1.0
In-Reply-To: 
 <CAH29n6PRPCj6viYdLD1fTC2c6G+QbREfwY7AU4R8zPb6TkqNPw@mail.gmail.com>
References: 
 <CAH29n6MYwbimcEpsEgp1JtdUcqSt5niROVT9Ze4rifZW8uaSWQ@mail.gmail.com>
 <20131105183828.GA13828@mafr.de>
 <BLU0-SMTP32305F3C9597F9E6120AB3EADF30@phx.gbl>
 <BLU0-SMTP51C1F8946BF8A6755986D4ADF30@phx.gbl>
 <CAA5C_pvNtK_iCZ5hKtR-X9BgL5Db_F6zgFpeh91eeXDVZwD5tA@mail.gmail.com>
 <CAH29n6NiUOF+9AK_twEBO1J328Z1J28MYhv-Pj37jDCOMTWtFA@mail.gmail.com>
 <20131107171425.GA2685@mafr.de>
 <CAH29n6PRPCj6viYdLD1fTC2c6G+QbREfwY7AU4R8zPb6TkqNPw@mail.gmail.com>
From: Chao Shi <stepinto@live.com>
Date: Fri, 8 Nov 2013 14:48:59 +0800
Subject: Re: [VOTE] Release Apache Crunch 0.8.0 RC0
To: dev@crunch.apache.org
Content-Type: multipart/alternative; boundary="001a11c37be41b711904eaa4c839"

--001a11c37be41b711904eaa4c839
Content-Type: text/plain; charset="ISO-8859-1"

+1

I verified things listed on Matthias's checklist and successfully built and
ran tests on both hadoop-1 and hadoop-2.


2013/11/8 Josh Wills <jwills@cloudera.com>

> On Thu, Nov 7, 2013 at 9:14 AM, Matthias Friedrich <matt@mafr.de> wrote:
>
> > Yup, that sounds plausible. In any case, even with a sporadically
> > failing integration test, the release is still valid from a legal
> > point of view. It's up to the release manager to decide whether
> > we need to restart the ceremony :)
> >
>
> It's certainly annoying and something to fix for the next release, but no,
> I don't think it merits a new RC.
>
>
> >
> > Regards,
> >   Matthias
> >
> > On Thursday, 2013-11-07, Josh Wills wrote:
> > > That does sound plausible-- I actually just ran into that issue on
> > another
> > > project I'm working on (i.e., like yesterday.)
> > >
> > >
> > > On Thu, Nov 7, 2013 at 8:23 AM, Gabriel Reid <gabriel.reid@gmail.com>
> > wrote:
> > >
> > > > On Thu, Nov 7, 2013 at 4:13 PM, Chao Shi <stepinto@live.com> wrote:
> > > > > I think this failure may depend on the execution order of tests,
> > because:
> > > > >
> > > > > 1) I can reproduce this failure on some source tree, but cannot on
> > > > another
> > > > > 2) I can only reproduce it with "mvn install", but "mvn test
> > > > > -Dtest=org.apache.crunch.CheckpointIT" cannot
> > > > > 3) I was trying to add some printf in the case, after saving the
> > file (I
> > > > > modified that CheckpointIT.java), I cannot reproduce it now.
> > > > >
> > > > > Does anyone have some clue?
> > > >
> > > > I have a feeling that this isn't due to order of execution. If it is
> > > > down to that, then as far as I can see it would mean that there is
> > > > something wrong with the TemporaryPath class, as that seems to be the
> > > > only thing that is shared between tests in that class.
> > > >
> > > > I'm actually wondering if this could be down to a timing issue in the
> > > > checkpointing code. The FileTargetImpl class checks if the
> > > > lastModified timestamp of the checkpointed file is greater than the
> > > > lastModified timestamp on the input source. Based on a check I just
> > > > did locally here on Mac OS X and on a Linux machine with ext4, the
> > > > time granularity of the lastModified timestamp on files is only
> > > > second-based, not millisecond-based. I'm thinking that this could be
> > > > an issue of the checkpointed file being created less than a second
> > > > after the startup of the test and creation of the source file
> > > > (although that seems really quick), and so the checkpointing wouldn't
> > > > think that the source file is older than the checkpointed file, so it
> > > > would run the full pipeline again (therefore resulting in two
> > > > execution stages instead of one.
> > > >
> > > > Does that sound plausible? If so, the only thing I can see that would
> > > > keep the code correct and prevent the test from failing would be to
> > > > add a Thread.sleep() into the test, but I hate having stuff like that
> > > > in tests.
> > > >
> > > > - Gabriel
> > > >
> > >
> > >
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>

--001a11c37be41b711904eaa4c839--