hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Date Sat, 20 Jun 2015 00:31:30 GMT
It seems that precommit-admin job which triggers the precommit-hbase job is
not running as frequently as it used to be. Only once every ~6 hours or so.

https://builds.apache.org/job/PreCommit-Admin/

The job is starved for the slaves.

Enis

On Fri, Jun 19, 2015 at 5:23 PM, Enis Söztutar <enis.soz@gmail.com> wrote:

> hadoopqa is acting weird for the last couple of days. It does not report
> back the results. One example is:
> https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console.
>
> I've excluded ubuntu3 from the list because it does not have protoc-2.5.
>
>
> On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <busbey@cloudera.com> wrote:
>
>> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds
>> to use branch-1.2. (ref HBASE-13912)
>>
>> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <busbey@cloudera.com> wrote:
>>
>> > I've add a new build test to run the ITs under sunny day conditions
>> using
>> > minicluster for both java 7 and java 8 on the 1.2 line.
>> >
>> > https://builds.apache.org/job/HBase-1.2-IT/
>> >
>> > I haven't turned on notifications yet, because BulkIngest and TestIngest
>> > are read. details on HBASE-13750
>> >
>> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> >
>> >> sigh. that should have been "is now a matrix build".
>> >>
>> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> >>
>> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8
>> in
>> >>> parallel.
>> >>>
>> >>> I saved the old job for now and left it disabled[2].
>> >>>
>> >>>
>> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> >>> [2]:
>> >>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>> >>>
>> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <busbey@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Sorry for the noise. I also updated the checkstyle step on
>> HBase-TRUNK
>> >>>>
>> >>>> from
>> >>>>     mvn checkstyle:checkstyle
>> >>>> to
>> >>>>     mvn -DskipTests package checkstyle:checkstyle
>> >>>>
>> >>>> to deal with the same issue. Looks all clear now.
>> >>>>
>> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <busbey@cloudera.com>
>> >>>> wrote:
>> >>>>
>> >>>>> Updated the following builds:
>> >>>>>
>> >>>>> * HBase-TRUNK
>> >>>>>
>> >>>>> moved from
>> >>>>>
>> >>>>>     mvn clean compile findbugs:findbugs
>> >>>>>
>> >>>>> to
>> >>>>>
>> >>>>>    mvn clean -DskipTests package findbugs:findbugs
>> >>>>>
>> >>>>> To work around the known issue where we can't do a bootstrap
compile
>> >>>>> without previous install or remote SNAPSHOT artifacts. (recently
>> triggered
>> >>>>> by the addition of the procedure module on master)
>> >>>>>
>> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <palomino219@gmail.com>
wrote:
>> >>>>>
>> >>>>>> Make findbugs and checkstyle plugins always run.
>> >>>>>> The default behavior only publishes result for stable builds,
but
>> >>>>>> master is
>> >>>>>> not always stable and sometimes we introduce new warnings
in
>> unstable
>> >>>>>> builds(see builds 6310-6314).
>> >>>>>> And findbugs and checkstyle will not fail unless we abort
the
>> building
>> >>>>>> process I think, so it is safe to turn it on every time.
>> >>>>>>
>> >>>>>> Thanks.
>> >>>>>>
>> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
>> >>>>>>
>> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report.
>> >>>>>> >
>> >>>>>> > Cheers
>> >>>>>> >
>> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <palomino219@gmail.com>
>> wrote:
>> >>>>>> >
>> >>>>>> > > Going to change the config of HBase-TRUNK jenkins
to show
>> >>>>>> findbugs,
>> >>>>>> > > checkstyle and jacoco coverage report.
>> >>>>>> > > The config has been tested on
>> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco
for nearly 30
>> >>>>>> times.
>> >>>>>> > Not
>> >>>>>> > > much different from HBase-TRUNK unless it runs
~30% slower(the
>> >>>>>> overhead
>> >>>>>> > of
>> >>>>>> > > collecting information for code coverage).
>> >>>>>> > > Thanks.
>> >>>>>> > >
>> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
>> >:
>> >>>>>> > >
>> >>>>>> > > > +1, thanks a lot for improving our build
hygiene.
>> >>>>>> > > >
>> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar
<
>> >>>>>> enis.soz@gmail.com>
>> >>>>>> > > wrote:
>> >>>>>> > > >
>> >>>>>> > > > > Thanks Sean. This is great.
>> >>>>>> > > > >
>> >>>>>> > > > > Enis
>> >>>>>> > > > >
>> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean
Busbey <
>> >>>>>> busbey@cloudera.com>
>> >>>>>> > > > wrote:
>> >>>>>> > > > >
>> >>>>>> > > > > > FYI I just finished chasing down
the breakage for mvn
>> site
>> >>>>>> on all
>> >>>>>> > > patch
>> >>>>>> > > > > > builds.
>> >>>>>> > > > > >
>> >>>>>> > > > > > HBASE-13191 consolidates the few
places in the test-patch
>> >>>>>> code
>> >>>>>> > where
>> >>>>>> > > we
>> >>>>>> > > > > > hard coded MAVEN_OPTS.
>> >>>>>> > > > > >
>> >>>>>> > > > > > If you look at the PreCommit job
now, we use the "set
>> >>>>>> environment
>> >>>>>> > > > > > variables" option to set MAVEN_OPTS
and then everything
>> else
>> >>>>>> > respects
>> >>>>>> > > > > that
>> >>>>>> > > > > > setting.
>> >>>>>> > > > > >
>> >>>>>> > > > > > I set the initial value to be a
combination of the memory
>> >>>>>> > limitations
>> >>>>>> > > > > we've
>> >>>>>> > > > > > been actually running with (the
~6G was getting ignored)
>> >>>>>> and the
>> >>>>>> > > > permgen
>> >>>>>> > > > > > needed for site.
>> >>>>>> > > > > >
>> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>> -XX:MaxPermSize=256m
>> >>>>>> > > > > >
>> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM,
Stack <
>> stack@duboce.net>
>> >>>>>> wrote:
>> >>>>>> > > > > >
>> >>>>>> > > > > > > I just made TRUNK and branch-1
builds use same jvm as
>> >>>>>> patch-build
>> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51
-- and I set the MAVEN_OPTS
>> to
>> >>>>>> be the
>> >>>>>> > > same
>> >>>>>> > > > as
>> >>>>>> > > > > > > those of trunk build too,
setting
>> >>>>>> MAVEN_OPTS="-Xmx6100m"... it
>> >>>>>> > had
>> >>>>>> > > > been
>> >>>>>> > > > > > > 3000.
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > Yours,
>> >>>>>> > > > > > > St.Ack
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22
PM, Stack <
>> stack@duboce.net>
>> >>>>>> wrote:
>> >>>>>> > > > > > >
>> >>>>>> > > > > > > > I upped hadoopqa retention
to keep last 100 builds
>> and
>> >>>>>> or last
>> >>>>>> > 7
>> >>>>>> > > > > days,
>> >>>>>> > > > > > > > whichever comes first.
>> >>>>>> > > > > > > > St.Ack
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at
9:38 AM, Stack <
>> stack@duboce.net
>> >>>>>> >
>> >>>>>> > wrote:
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > > >> Branch-1 and master
have stabilized and now run
>> mostly
>> >>>>>> blue
>> >>>>>> > > (give
>> >>>>>> > > > or
>> >>>>>> > > > > > > take
>> >>>>>> > > > > > > >> the odd failure)
[1][2]. Having a mostly blue
>> branch-1
>> >>>>>> has
>> >>>>>> > > helped
>> >>>>>> > > > us
>> >>>>>> > > > > > > >> identify at least
one destabilizing commit in the
>> last
>> >>>>>> few
>> >>>>>> > days,
>> >>>>>> > > > > maybe
>> >>>>>> > > > > > > two;
>> >>>>>> > > > > > > >> this is as it should
be (smile).
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> Lets keep our builds
blue. If you commit a patch,
>> make
>> >>>>>> sure
>> >>>>>> > > > > subsequent
>> >>>>>> > > > > > > >> builds stay blue.
You can subscribe to
>> >>>>>> > builds@hbase.apache.org
>> >>>>>> > > to
>> >>>>>> > > > > get
>> >>>>>> > > > > > > >> notice of failures
if not already subscribed.
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> Thanks,
>> >>>>>> > > > > > > >> St.Ack
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> 1.
>> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> >>>>>> > > > > > > >> 2.
>> >>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >> On Mon, Oct 13, 2014
at 4:41 PM, Stack <
>> >>>>>> stack@duboce.net>
>> >>>>>> > > wrote:
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >>> A few notes on
testing.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Too long to read,
infra is more capable now and
>> after
>> >>>>>> some
>> >>>>>> > > work,
>> >>>>>> > > > we
>> >>>>>> > > > > > are
>> >>>>>> > > > > > > >>> seeing branch-1
and trunk mostly running blue. Lets
>> >>>>>> try and
>> >>>>>> > > keep
>> >>>>>> > > > it
>> >>>>>> > > > > > > this
>> >>>>>> > > > > > > >>> way going forward.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Apache Infra
has new, more capable hardware.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> A recent spurt
of test fixing combined with more
>> >>>>>> capable
>> >>>>>> > > hardware
>> >>>>>> > > > > > seems
>> >>>>>> > > > > > > >>> to have gotten
us to a new place; tests are mostly
>> >>>>>> passing
>> >>>>>> > now
>> >>>>>> > > on
>> >>>>>> > > > > > > branch-1
>> >>>>>> > > > > > > >>> and master. 
Lets try and keep it this way and
>> start
>> >>>>>> to trust
>> >>>>>> > > our
>> >>>>>> > > > > > test
>> >>>>>> > > > > > > runs
>> >>>>>> > > > > > > >>> again.  Just
a few flakies remain.  Lets try and
>> nail
>> >>>>>> them.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Our tests now
run in parallel with other test
>> suites
>> >>>>>> where
>> >>>>>> > > > previous
>> >>>>>> > > > > > we
>> >>>>>> > > > > > > >>> ran alone. You
can see this sometimes when our
>> zombie
>> >>>>>> > detector
>> >>>>>> > > > > > reports
>> >>>>>> > > > > > > >>> tests from another
project altogether as lingerers
>> >>>>>> (To be
>> >>>>>> > > fixed).
>> >>>>>> > > > > > > Some of
>> >>>>>> > > > > > > >>> our tests are
failing because a concurrent hbase
>> run
>> >>>>>> is
>> >>>>>> > undoing
>> >>>>>> > > > > > > classes and
>> >>>>>> > > > > > > >>> data from under
it. Also, lets fix.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Our tests are
brittle. It takes 75minutes for them
>> to
>> >>>>>> > complete.
>> >>>>>> > > > > Many
>> >>>>>> > > > > > > >>> are heavy-duty
integration tests starting up
>> multiple
>> >>>>>> > clusters
>> >>>>>> > > > and
>> >>>>>> > > > > > > >>> mapreduce all
in the one JVM. It is a miracle they
>> >>>>>> pass at
>> >>>>>> > all.
>> >>>>>> > > > > > > Usually
>> >>>>>> > > > > > > >>> integration tests
have been cast as unit tests
>> >>>>>> because there
>> >>>>>> > > was
>> >>>>>> > > > no
>> >>>>>> > > > > > > where
>> >>>>>> > > > > > > >>> else for them
to get an airing.  We have the
>> hbase-it
>> >>>>>> suite
>> >>>>>> > now
>> >>>>>> > > > > which
>> >>>>>> > > > > > > would
>> >>>>>> > > > > > > >>> be a more apt
place but until these are run on a
>> >>>>>> regular
>> >>>>>> > basis
>> >>>>>> > > in
>> >>>>>> > > > > > > public
>> >>>>>> > > > > > > >>> for all to see,
the fat integration tests disguised
>> >>>>>> as unit
>> >>>>>> > > tests
>> >>>>>> > > > > > will
>> >>>>>> > > > > > > >>> remain.  A review
of our current unit tests weeding
>> >>>>>> the old
>> >>>>>> > > cruft
>> >>>>>> > > > > and
>> >>>>>> > > > > > > the
>> >>>>>> > > > > > > >>> no longer relevant
or duplicates would be a nice
>> >>>>>> undertaking
>> >>>>>> > if
>> >>>>>> > > > > > > someone is
>> >>>>>> > > > > > > >>> looking to contribute.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> Alex Newman has
been working on making our tests
>> work
>> >>>>>> up on
>> >>>>>> > > > travis
>> >>>>>> > > > > > and
>> >>>>>> > > > > > > >>> circle-ci.  That'll
be sweet when it goes
>> >>>>>> end-to-end.  He
>> >>>>>> > also
>> >>>>>> > > > > added
>> >>>>>> > > > > > in
>> >>>>>> > > > > > > >>> some "type" categorizations
-- client, filter,
>> >>>>>> mapreduce --
>> >>>>>> > > > > alongside
>> >>>>>> > > > > > > our
>> >>>>>> > > > > > > >>> old "sizing"
categorizations of small/medium/large.
>> >>>>>> His
>> >>>>>> > > thinking
>> >>>>>> > > > > is
>> >>>>>> > > > > > > that
>> >>>>>> > > > > > > >>> we can run these
categorizations in parallel so we
>> >>>>>> could run
>> >>>>>> > > the
>> >>>>>> > > > > > total
>> >>>>>> > > > > > > >>> suite in about
the time of the longest test, say
>> >>>>>> > 20-30minutes?
>> >>>>>> > > > We
>> >>>>>> > > > > > > could
>> >>>>>> > > > > > > >>> even change Apache
to run them this way.
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>> FYI,
>> >>>>>> > > > > > > >>> St.Ack
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>>
>> >>>>>> > > > > > > >>
>> >>>>>> > > > > > > >
>> >>>>>> > > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > >
>> >>>>>> > > > > > --
>> >>>>>> > > > > > Sean
>> >>>>>> > > > > >
>> >>>>>> > > > >
>> >>>>>> > > >
>> >>>>>> > > >
>> >>>>>> > > >
>> >>>>>> > > > --
>> >>>>>> > > > Best regards,
>> >>>>>> > > >
>> >>>>>> > > >    - Andy
>> >>>>>> > > >
>> >>>>>> > > > Problems worthy of attack prove their worth
by hitting back.
>> -
>> >>>>>> Piet
>> >>>>>> > Hein
>> >>>>>> > > > (via Tom White)
>> >>>>>> > > >
>> >>>>>> > >
>> >>>>>> >
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sean
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Sean
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sean
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Sean
>> >>
>> >
>> >
>> >
>> > --
>> > Sean
>> >
>>
>>
>>
>> --
>> Sean
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message