spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-5053) Test maintenance branches on Jenkins using SBT
Date Fri, 09 Jan 2015 20:03:35 GMT

    [ https://issues.apache.org/jira/browse/SPARK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271793#comment-14271793
] 

Josh Rosen commented on SPARK-5053:
-----------------------------------

It doesn't look like the "disable web UI in tests" patches were backported to {{branch-1.0}},
so that's probably a large part of the problem.

In the case of the ThriftServer2 tests, I guess this is another case where our trick to disable
the UI in tests did not apply to a process that was launched through shell scripts, so we
may need to apply a fix that's similar to what I did for the SparkSubmitSuite tests.

It looks like there's some additional sources of flakiness in {{branch-1.2}}, though.  For
example, "org.apache.spark.metrics.InputOutputMetricsSuite.input metrics when reading text
file with multiple splits" has failed on-and-off for a few builds:

https://amplab.cs.berkeley.edu/jenkins/job/Spark-1.2-SBT/5/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=centos/testReport/junit/org.apache.spark.metrics/InputOutputMetricsSuite/input_metrics_when_reading_text_file_with_multiple_splits/

> Test maintenance branches on Jenkins using SBT
> ----------------------------------------------
>
>                 Key: SPARK-5053
>                 URL: https://issues.apache.org/jira/browse/SPARK-5053
>             Project: Spark
>          Issue Type: New Feature
>          Components: Project Infra
>            Reporter: Josh Rosen
>            Priority: Blocker
>
> We need to create Jenkins jobs to test maintenance branches using SBT.  The current Maven
jobs for backport branches do not run the same checks that the pull request builder / SBT
builds do (e.g. MiMa checks, PySpark, RAT, etc.) which means that cherry-picking backports
can silently break things and we'll only discover it once PRs that are explicitly opened against
those branches fail tests; this long delay between introducing test failures and detecting
them is a huge productivity issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message