hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张铎 <palomino...@gmail.com>
Subject Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Date Tue, 31 Mar 2015 01:49:39 GMT
Make findbugs and checkstyle plugins always run.
The default behavior only publishes result for stable builds, but master is
not always stable and sometimes we introduce new warnings in unstable
builds(see builds 6310-6314).
And findbugs and checkstyle will not fail unless we abort the building
process I think, so it is safe to turn it on every time.

Thanks.

2015-03-22 22:44 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:

> +1 on letting HBase-TRUNK jenkins show coverage report.
>
> Cheers
>
> On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <palomino219@gmail.com> wrote:
>
> > Going to change the config of HBase-TRUNK jenkins to show findbugs,
> > checkstyle and jacoco coverage report.
> > The config has been tested on
> > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 times.
> Not
> > much different from HBase-TRUNK unless it runs ~30% slower(the overhead
> of
> > collecting information for code coverage).
> > Thanks.
> >
> > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org>:
> >
> > > +1, thanks a lot for improving our build hygiene.
> > >
> > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar <enis.soz@gmail.com>
> > wrote:
> > >
> > > > Thanks Sean. This is great.
> > > >
> > > > Enis
> > > >
> > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey <busbey@cloudera.com>
> > > wrote:
> > > >
> > > > > FYI I just finished chasing down the breakage for mvn site on all
> > patch
> > > > > builds.
> > > > >
> > > > > HBASE-13191 consolidates the few places in the test-patch code
> where
> > we
> > > > > hard coded MAVEN_OPTS.
> > > > >
> > > > > If you look at the PreCommit job now, we use the "set environment
> > > > > variables" option to set MAVEN_OPTS and then everything else
> respects
> > > > that
> > > > > setting.
> > > > >
> > > > > I set the initial value to be a combination of the memory
> limitations
> > > > we've
> > > > > been actually running with (the ~6G was getting ignored) and the
> > > permgen
> > > > > needed for site.
> > > > >
> > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m
> > > > >
> > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <stack@duboce.net>
wrote:
> > > > >
> > > > > > I just made TRUNK and branch-1 builds use same jvm as patch-build
> > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be
the
> > same
> > > as
> > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"...
it
> had
> > > been
> > > > > > 3000.
> > > > > >
> > > > > > Yours,
> > > > > > St.Ack
> > > > > >
> > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <stack@duboce.net>
wrote:
> > > > > >
> > > > > > > I upped hadoopqa retention to keep last 100 builds and
or last
> 7
> > > > days,
> > > > > > > whichever comes first.
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <stack@duboce.net>
> wrote:
> > > > > > >
> > > > > > >> Branch-1 and master have stabilized and now run mostly
blue
> > (give
> > > or
> > > > > > take
> > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1
has
> > helped
> > > us
> > > > > > >> identify at least one destabilizing commit in the last
few
> days,
> > > > maybe
> > > > > > two;
> > > > > > >> this is as it should be (smile).
> > > > > > >>
> > > > > > >> Lets keep our builds blue. If you commit a patch, make
sure
> > > > subsequent
> > > > > > >> builds stay blue. You can subscribe to
> builds@hbase.apache.org
> > to
> > > > get
> > > > > > >> notice of failures if not already subscribed.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> St.Ack
> > > > > > >>
> > > > > > >> 1.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
> > > > > > >> 2.
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
> > > > > > >>
> > > > > > >>
> > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <stack@duboce.net>
> > wrote:
> > > > > > >>
> > > > > > >>> A few notes on testing.
> > > > > > >>>
> > > > > > >>> Too long to read, infra is more capable now and
after some
> > work,
> > > we
> > > > > are
> > > > > > >>> seeing branch-1 and trunk mostly running blue.
Lets try and
> > keep
> > > it
> > > > > > this
> > > > > > >>> way going forward.
> > > > > > >>>
> > > > > > >>> Apache Infra has new, more capable hardware.
> > > > > > >>>
> > > > > > >>> A recent spurt of test fixing combined with more
capable
> > hardware
> > > > > seems
> > > > > > >>> to have gotten us to a new place; tests are mostly
passing
> now
> > on
> > > > > > branch-1
> > > > > > >>> and master.  Lets try and keep it this way and
start to trust
> > our
> > > > > test
> > > > > > runs
> > > > > > >>> again.  Just a few flakies remain.  Lets try and
nail them.
> > > > > > >>>
> > > > > > >>> Our tests now run in parallel with other test suites
where
> > > previous
> > > > > we
> > > > > > >>> ran alone. You can see this sometimes when our
zombie
> detector
> > > > > reports
> > > > > > >>> tests from another project altogether as lingerers
(To be
> > fixed).
> > > > > > Some of
> > > > > > >>> our tests are failing because a concurrent hbase
run is
> undoing
> > > > > > classes and
> > > > > > >>> data from under it. Also, lets fix.
> > > > > > >>>
> > > > > > >>> Our tests are brittle. It takes 75minutes for them
to
> complete.
> > > > Many
> > > > > > >>> are heavy-duty integration tests starting up multiple
> clusters
> > > and
> > > > > > >>> mapreduce all in the one JVM. It is a miracle they
pass at
> all.
> > > > > > Usually
> > > > > > >>> integration tests have been cast as unit tests
because there
> > was
> > > no
> > > > > > where
> > > > > > >>> else for them to get an airing.  We have the hbase-it
suite
> now
> > > > which
> > > > > > would
> > > > > > >>> be a more apt place but until these are run on
a regular
> basis
> > in
> > > > > > public
> > > > > > >>> for all to see, the fat integration tests disguised
as unit
> > tests
> > > > > will
> > > > > > >>> remain.  A review of our current unit tests weeding
the old
> > cruft
> > > > and
> > > > > > the
> > > > > > >>> no longer relevant or duplicates would be a nice
undertaking
> if
> > > > > > someone is
> > > > > > >>> looking to contribute.
> > > > > > >>>
> > > > > > >>> Alex Newman has been working on making our tests
work up on
> > > travis
> > > > > and
> > > > > > >>> circle-ci.  That'll be sweet when it goes end-to-end.
 He
> also
> > > > added
> > > > > in
> > > > > > >>> some "type" categorizations -- client, filter,
mapreduce --
> > > > alongside
> > > > > > our
> > > > > > >>> old "sizing" categorizations of small/medium/large.
 His
> > thinking
> > > > is
> > > > > > that
> > > > > > >>> we can run these categorizations in parallel so
we could run
> > the
> > > > > total
> > > > > > >>> suite in about the time of the longest test, say
> 20-30minutes?
> > > We
> > > > > > could
> > > > > > >>> even change Apache to run them this way.
> > > > > > >>>
> > > > > > >>> FYI,
> > > > > > >>> St.Ack
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sean
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message