hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: we need a fix: precommit failures correlate to hdfs patches
Date Mon, 04 May 2015 04:02:57 GMT
The patch artifact directory in the mainline hadoop jenkins jobs are
outside of the workspace. I'm not sure what, if anything, jenkins
guarantees about files out of the main workspace.

They all write to ${WORKSPACE}/../patchProcess, which will probably collide
if multiple runs happen on the same machine. They also all blindly move
that directory at the end of the run.

On Sun, May 3, 2015 at 3:02 PM, Allen Wittenauer <aw@altiscale.com> wrote:

>
>         So, as some may have noticed, I slammed the Jenkins servers over
> the weekend to get some recent patch test runs in JIRA for the bug bash
> this week.  I've had a suspicion for a while now that either the long run
> times of the hadoop-hdfs module unit tests (typically 2+ hours) or the hdfs
> tests themselves were related to the patch process directory getting
> removed out from underneath test-patch.
>
>         To test the hypothesis, I submitted all of the non-HDFS patches so
> that they were first in the queue.  Let them run for a very long time.
> Jenkins bounced back and forth between YARN, MR, and HADOOP.   No issues
> encounters.  Added HDFS patches into the mix. BOOM. The dreaded "The patch
> artifact directory has been removed! “ started to appear here and there.
> This seems to provide some evidence that, yes, hdfs unit tests are
> directory or indirectly related to the failures.
>
>         IMO, I think we need to take a serious look at:
>
>         * splitting up the hadoop-hdfs module into multiple modules to
> reduce unit test run times
>         * checking to see if the pre commit hooks in hdfs are different
> than the rest (I do know that the YARN bits are different and appear to
> have some bugs as well)
>         * increasing the timeout for jenkins job runs
>
>         FWIW, I’ve also found some minor things here and there with the
> rewritten test-patch.sh.  JIRAs have been filed.  One critical, one major
> and a handful of minor things.




-- 
Sean

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message