spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <>
Subject [jira] [Commented] (SPARK-5053) Test maintenance branches on Jenkins using SBT
Date Fri, 09 Jan 2015 20:03:35 GMT


Josh Rosen commented on SPARK-5053:

It doesn't look like the "disable web UI in tests" patches were backported to {{branch-1.0}},
so that's probably a large part of the problem.

In the case of the ThriftServer2 tests, I guess this is another case where our trick to disable
the UI in tests did not apply to a process that was launched through shell scripts, so we
may need to apply a fix that's similar to what I did for the SparkSubmitSuite tests.

It looks like there's some additional sources of flakiness in {{branch-1.2}}, though.  For
example, "org.apache.spark.metrics.InputOutputMetricsSuite.input metrics when reading text
file with multiple splits" has failed on-and-off for a few builds:,label=centos/testReport/junit/org.apache.spark.metrics/InputOutputMetricsSuite/input_metrics_when_reading_text_file_with_multiple_splits/

> Test maintenance branches on Jenkins using SBT
> ----------------------------------------------
>                 Key: SPARK-5053
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Project Infra
>            Reporter: Josh Rosen
>            Priority: Blocker
> We need to create Jenkins jobs to test maintenance branches using SBT.  The current Maven
jobs for backport branches do not run the same checks that the pull request builder / SBT
builds do (e.g. MiMa checks, PySpark, RAT, etc.) which means that cherry-picking backports
can silently break things and we'll only discover it once PRs that are explicitly opened against
those branches fail tests; this long delay between introducing test failures and detecting
them is a huge productivity issue.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message