hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Date Mon, 07 Mar 2016 18:27:59 GMT
FYI, I've just updated our precommit jobs to use the 0.2.0 release of Yetus
that came out today.

After keeping an eye out for strangeness today I'll turn docker mode back
on by default tonight.

On Wed, Jan 13, 2016 at 10:14 AM, Sean Busbey <busbey@apache.org> wrote:

> FYI, I added a new parameter to the precommit job:
>
> * USE_YETUS_PRERELEASE - causes us to use the HEAD of the apache/yetus
> repo rather than our chosen release
>
> It defaults to inactive, but can be used in manually-triggered runs to
> test a solution to a problem in the yetus library. At the moment, I'm
> using it to test a solution to default module ordering  as seen in
> HBASE-15075.
>
> On Fri, Jan 8, 2016 at 7:58 AM, Sean Busbey <busbey@cloudera.com> wrote:
> > FYI, I just pushed HBASE-13525 (switch to Apache Yetus for precommit
> tests)
> > and updated our jenkins precommit build to use it.
> >
> > Jenkins job has some explanation:
> >
> https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-HBASE-Build/
> >
> > Release note from HBASE-13525 does as well.
> >
> > The old job will stick around here for a couple of weeks, in case we need
> > to refer back to it:
> >
> >
> https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-HBASE-Build-deprecated/
> >
> > If something looks awry, please drop a note on HBASE-13525 while it
> remains
> > open (and make a new issue after).
> >
> >
> > On Wed, Dec 2, 2015 at 3:22 PM, Stack <stack@duboce.net> wrote:
> >
> >> As part of my continuing advocacy of builds.apache.org and that their
> >> results are now worthy of our trust and nurture, here are some
> highlights
> >> from the last few days of builds:
> >>
> >> + hadoopqa is now finding zombies before the patch is committed.
> >> HBASE-14888 showed "-1 core tests. The patch failed these unit tests:"
> but
> >> didn't have any failed tests listed (I'm trying to see if I can do
> anything
> >> about this...). Running our little ./dev-tools/findHangingTests.py
> against
> >> the consoleText, it showed a hanging test. Running locally, I see same
> >> hang. This is before the patch landed.
> >> + Our branch runs are now near totally zombie and flakey free -- still
> some
> >> work to do -- but a recent patch that seemed harmless was causing a
> >> reliable flake fail in the backport to branch-1* confirmed by local
> runs.
> >> The flakeyness was plain to see up in builds.apache.org.
> >> + In the last few days I've committed a patch that included javadoc
> >> warnings even though hadoopqa said the patch introduced javadoc issues
> (I
> >> missed it). This messed up life for folks subsequently as their patches
> now
> >> reported javadoc issues....
> >>
> >> In short, I suggest that builds.apache.org is worth keeping an eye on,
> >> make
> >> sure you get a clean build out of hadoopqa before committing anything,
> and
> >> lets all work together to try and keep our builds blue: it'll save us
> all
> >> work in the long run.
> >>
> >> St.Ack
> >>
> >>
> >> On Tue, Nov 4, 2014 at 9:38 AM, Stack <stack@duboce.net> wrote:
> >>
> >> > Branch-1 and master have stabilized and now run mostly blue (give or
> take
> >> > the odd failure) [1][2]. Having a mostly blue branch-1 has helped us
> >> > identify at least one destabilizing commit in the last few days, maybe
> >> two;
> >> > this is as it should be (smile).
> >> >
> >> > Lets keep our builds blue. If you commit a patch, make sure subsequent
> >> > builds stay blue. You can subscribe to builds@hbase.apache.org to get
> >> > notice of failures if not already subscribed.
> >> >
> >> > Thanks,
> >> > St.Ack
> >> >
> >> > 1. https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> >> > 2. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> >> >
> >> >
> >> > On Mon, Oct 13, 2014 at 4:41 PM, Stack <stack@duboce.net> wrote:
> >> >
> >> >> A few notes on testing.
> >> >>
> >> >> Too long to read, infra is more capable now and after some work, we
> are
> >> >> seeing branch-1 and trunk mostly running blue. Lets try and keep it
> this
> >> >> way going forward.
> >> >>
> >> >> Apache Infra has new, more capable hardware.
> >> >>
> >> >> A recent spurt of test fixing combined with more capable hardware
> seems
> >> >> to have gotten us to a new place; tests are mostly passing now on
> >> branch-1
> >> >> and master.  Lets try and keep it this way and start to trust our
> test
> >> runs
> >> >> again.  Just a few flakies remain.  Lets try and nail them.
> >> >>
> >> >> Our tests now run in parallel with other test suites where previous
> we
> >> >> ran alone. You can see this sometimes when our zombie detector
> reports
> >> >> tests from another project altogether as lingerers (To be fixed).
> Some
> >> of
> >> >> our tests are failing because a concurrent hbase run is undoing
> classes
> >> and
> >> >> data from under it. Also, lets fix.
> >> >>
> >> >> Our tests are brittle. It takes 75minutes for them to complete.  Many
> >> are
> >> >> heavy-duty integration tests starting up multiple clusters and
> mapreduce
> >> >> all in the one JVM. It is a miracle they pass at all.  Usually
> >> integration
> >> >> tests have been cast as unit tests because there was no where else
> for
> >> them
> >> >> to get an airing.  We have the hbase-it suite now which would be a
> more
> >> apt
> >> >> place but until these are run on a regular basis in public for all
to
> >> see,
> >> >> the fat integration tests disguised as unit tests will remain.  A
> >> review of
> >> >> our current unit tests weeding the old cruft and the no longer
> relevant
> >> or
> >> >> duplicates would be a nice undertaking if someone is looking to
> >> contribute.
> >> >>
> >> >> Alex Newman has been working on making our tests work up on travis
> and
> >> >> circle-ci.  That'll be sweet when it goes end-to-end.  He also added
> in
> >> >> some "type" categorizations -- client, filter, mapreduce -- alongside
> >> our
> >> >> old "sizing" categorizations of small/medium/large.  His thinking is
> >> that
> >> >> we can run these categorizations in parallel so we could run the
> total
> >> >> suite in about the time of the longest test, say 20-30minutes?  We
> could
> >> >> even change Apache to run them this way.
> >> >>
> >> >> FYI,
> >> >> St.Ack
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >>
> >
> >
> >
> > --
> > Sean
>



-- 
busbey

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message