spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <skn...@berkeley.edu>
Subject Re: Unit test logs in Jenkins?
Date Thu, 02 Apr 2015 15:59:37 GMT
i agree with all of this.  but can we please break up the tests and make
them shorter?  :)

On Thu, Apr 2, 2015 at 8:54 AM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:

> This is secondary to Marcelo’s question, but I wanted to comment on this:
>
> Its main limitation is more cultural than technical: you need to get people
> to care about intermittent test runs, otherwise you can end up with
> failures that nobody keeps on top of
>
> This is a problem that plagues Spark as well, but there *is* a technical
> solution.
>
> The solution is simple: *All* the builds that we care about run for *every*
> proposed change. If *any* build fails, the change doesn’t make it into the
> repository.
>
> Spark already has a pull request builder that tests and reports back on
> PRs. Committers don’t merge in PRs when this builder reports that it failed
> some tests. That’s a good thing.
>
> The problem is that there are several other builds that we run on a fixed
> interval, independent of the pull request builder. These builds test
> different configurations, dependency versions, and environments than what
> the PR builder covers. If one of those builds fails, it fails on its own
> little island, with no-one to hear it scream. The build failure is detached
> from the PR that caused it to fail.
>
> What should happen is that the whole matrix of stuff we care to test gets
> run for every PR. No PR goes in if any build we care about fails for that
> PR, and every build we care about runs for every commit of every PR.
>
> Really, this is just an extension of the basic idea of the PR builder. It
> doesn’t make much sense to test stuff *after* it has been committed and
> potentially broken things. And it becomes exponentially more difficult to
> find and fix a problem the longer it has been festering in the repo. It’s
> best to keep such problems out in the first place.
>
> With some more work on our CI infrastructure, I think this can be done.
> Maybe even later this year.
>
> Nick
>
> On Thu, Apr 2, 2015 at 6:02 AM Steve Loughran stevel@hortonworks.com
> <http://mailto:stevel@hortonworks.com> wrote:
>
>
> > > On 2 Apr 2015, at 06:31, Patrick Wendell <pwendell@gmail.com> wrote:
> > >
> > > Hey Marcelo,
> > >
> > > Great question. Right now, some of the more active developers have an
> > > account that allows them to log into this cluster to inspect logs (we
> > > copy the logs from each run to a node on that cluster). The
> > > infrastructure is maintained by the AMPLab.
> > >
> > > I will put you in touch the someone there who can get you an account.
> > >
> > > This is a short term solution. The longer term solution is to have
> > > these scp'd regularly to an S3 bucket or somewhere people can get
> > > access to them, but that's not ready yet.
> > >
> > > - Patrick
> > >
> > >>
> >
> >
> > ASF Jenkins is always there to play with; committers/PMC members should
> > just need to file a BUILD JIRA to get access.
> >
> > Its main limitation is more cultural than technical: you need to get
> > people to care about intermittent test runs, otherwise you can end up
> with
> > failures that nobody keeps on top of
> > https://builds.apache.org/view/H-L/view/Hadoop/
> >
> > Someone really needs to own the "keep the builds working" problem -and
> > have the ability to somehow kick others into fixing things. The latter is
> > pretty hard cross-organisation
> >
> >
> > >> That would be really helpful to debug build failures. The scalatest
> > >> output isn't all that helpful.
> > >>
> >
> > Potentially an issue with the test runner, rather than the tests
> > themselves.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> > For additional commands, e-mail: dev-help@spark.apache.org
> >
> >  ​
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message