spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Recent Spark test failures
Date Fri, 15 May 2015 17:30:54 GMT
You all are looking only at the pull request builder. It just does one
build to sanity-check a pull request, since that already takes 2 hours and
would be prohibitive to build all configurations for every push. There is a
different set of Jenkins jobs that periodically tests master against a lot
more configurations, including Hadoop 2.4.

On Fri, May 15, 2015 at 6:02 PM, Frederick R Reiss <frreiss@us.ibm.com>
wrote:

> The PR builder seems to be building against Hadoop 2.3. In the log for the
> most recent successful build (
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32805/consoleFull
> ) I see:
>
> =========================================================================
> Building Spark
> =========================================================================
> [info] Compile with Hive 0.13.1
> [info] Building Spark with these arguments: -Pyarn -Phadoop-2.3
> -Dhadoop.version=2.3.0 -Pkinesis-asl -Phive -Phive-thriftserver
> ...
> =========================================================================
> Running Spark unit tests
> =========================================================================
> [info] Running Spark tests with these arguments: -Pyarn -Phadoop-2.3
> -Dhadoop.version=2.3.0 -Pkinesis-asl test
>
> Is anyone testing individual pull requests against Hadoop 2.4 or 2.6
> before the code is declared "clean"?
>
> Fred
>
> [image: Inactive hide details for Ted Yu ---05/15/2015 09:29:09
> AM---Jenkins build against hadoop 2.4 has been unstable recently: https]Ted
> Yu ---05/15/2015 09:29:09 AM---Jenkins build against hadoop 2.4 has been
> unstable recently: https://amplab.cs.berkeley.edu/jenkins/
>
> From: Ted Yu <yuzhihong@gmail.com>
> To: Andrew Or <andrew@databricks.com>
> Cc: "dev@spark.apache.org" <dev@spark.apache.org>
> Date: 05/15/2015 09:29 AM
> Subject: Re: Recent Spark test failures
> ------------------------------
>
>
>
> Jenkins build against hadoop 2.4 has been unstable recently:
>
> *https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=hadoop-2.4,label=centos/*
> <https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=hadoop-2.4,label=centos/>
>
> I haven't found the test which hung / failed in recent Jenkins builds.
>
> But PR builder has several green builds lately:
> *https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/*
> <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/>
>
> Maybe PR builder doesn't build against hadoop 2.4 ?
>
> Cheers
>
> On Mon, May 11, 2015 at 1:11 PM, Ted Yu <*yuzhihong@gmail.com*
> <yuzhihong@gmail.com>> wrote:
>
>    Makes sense.
>
>    Having high determinism in these tests would make Jenkins build stable.
>
>    On Mon, May 11, 2015 at 1:08 PM, Andrew Or <*andrew@databricks.com*
>    <andrew@databricks.com>> wrote:
>       Hi Ted,
>
>       Yes, those two options can be useful, but in general I think the
>       standard to set is that tests should never fail. It's actually the worst if
>       tests fail sometimes but not others, because we can't reproduce them
>       deterministically. Using -M and -A actually tolerates flaky tests to a
>       certain extent, and I would prefer to instead increase the determinism in
>       these tests.
>
>       -Andrew
>
>       2015-05-08 17:56 GMT-07:00 Ted Yu <*yuzhihong@gmail.com*
>       <yuzhihong@gmail.com>>:
>       Andrew:
>          Do you think the -M and -A options described here can be used in
>          test runs ?
>          *http://scalatest.org/user_guide/using_the_runner*
>          <http://scalatest.org/user_guide/using_the_runner>
>
>          Cheers
>
>          On Wed, May 6, 2015 at 5:41 PM, Andrew Or <
>          *andrew@databricks.com* <andrew@databricks.com>> wrote:
>             Dear all,
>
>             I'm sure you have all noticed that the Spark tests have been
>             fairly
>             unstable recently. I wanted to share a tool that I use to
>             track which tests
>             have been failing most often in order to prioritize fixing
>             these flaky
>             tests.
>
>             Here is an output of the tool. This spreadsheet reports the
>             top 10 failed
>             tests this week (ending yesterday 5/5):
>
>             *https://docs.google.com/spreadsheets/d/1Iv_UDaTFGTMad1sOQ_s4ddWr6KD3PuFIHmTSzL7LSb4*
>             <https://docs.google.com/spreadsheets/d/1Iv_UDaTFGTMad1sOQ_s4ddWr6KD3PuFIHmTSzL7LSb4>
>
>             It is produced by a small project:
> *https://github.com/andrewor14/spark-test-failures*
>             <https://github.com/andrewor14/spark-test-failures>
>
>             I have been filing JIRAs on flaky tests based on this tool.
>             Hopefully we
>             can collectively stabilize the build a little more as we near
>             the release
>             for Spark 1.4.
>
>             -Andrew
>
>
>
>

Mime
View raw message