hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Date Tue, 03 Nov 2015 14:44:43 GMT
Stuff seems to be basically working.  I just committed a fix for this
message so it should be gone now:

grep: unknown devices method
grep: write error: Broken pipe


Hopefully we've seen last of the errant surefire kills.

We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We should
get rid of the former now?

St.Ack

On Mon, Nov 2, 2015 at 11:03 PM, Stack <stack@duboce.net> wrote:

> I just did an edit of all of our jenkins configuration. I changed the
> post-build zombie section. The surefire-killer was hanging out here it
> seems (For detail, see HBASE-14589 Looking for the surefire-killer; builds
> being killed...). I may have messed up builds. Will fix in morning if so
> (currently builds seem a bit backed up so won't wait around).
>
> St.Ack
>
> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <busbey@cloudera.com> wrote:
>
>> Hi Folks,
>>
>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS]
>> reducing our load on builds.a.o' on reasoning or if you have concerns.
>>
>> Our project job view now has a brief summary of what's going on:
>>
>> https://builds.apache.org/view/H-L/view/HBase/
>>
>> * Where we had matrix builds, I confirmed they're doing the same thing
>> (in at least one test-configuration) as the non-matrix version
>> * I disabled jobs that were duplicating work done in matrix jobs, or
>> that only worked on a tag
>> * Made sure all non-disabled 0.98+ jobs are set to notify jira +
>> builds mailing list
>> * I updated pre-1.1 jobs to look for changes 1/day
>> * I updated 1.1 jobs to look for changes every 8 hours
>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours
>>
>> There's still some non-pressing clean up work to do, so I'll file a
>> jira later today just to make sure it's all recorded somewhere.
>>
>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2
>> builds to
>> > use branch-1.2. (ref HBASE-13912)
>> >
>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> >>
>> >> I've add a new build test to run the ITs under sunny day conditions
>> using
>> >> minicluster for both java 7 and java 8 on the 1.2 line.
>> >>
>> >> https://builds.apache.org/job/HBase-1.2-IT/
>> >>
>> >> I haven't turned on notifications yet, because BulkIngest and
>> TestIngest
>> >> are read. details on HBASE-13750
>> >>
>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> >>>
>> >>> sigh. that should have been "is now a matrix build".
>> >>>
>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <busbey@cloudera.com>
>> wrote:
>> >>>>
>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and
JDK8
>> in
>> >>>> parallel.
>> >>>>
>> >>>> I saved the old job for now and left it disabled[2].
>> >>>>
>> >>>>
>> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/
>> >>>> [2]:
>> >>>>
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/
>> >>>>
>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <busbey@cloudera.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Sorry for the noise. I also updated the checkstyle step on
>> HBase-TRUNK
>> >>>>>
>> >>>>> from
>> >>>>>     mvn checkstyle:checkstyle
>> >>>>> to
>> >>>>>     mvn -DskipTests package checkstyle:checkstyle
>> >>>>>
>> >>>>> to deal with the same issue. Looks all clear now.
>> >>>>>
>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <busbey@cloudera.com>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Updated the following builds:
>> >>>>>>
>> >>>>>> * HBase-TRUNK
>> >>>>>>
>> >>>>>> moved from
>> >>>>>>
>> >>>>>>     mvn clean compile findbugs:findbugs
>> >>>>>>
>> >>>>>> to
>> >>>>>>
>> >>>>>>    mvn clean -DskipTests package findbugs:findbugs
>> >>>>>>
>> >>>>>> To work around the known issue where we can't do a bootstrap
>> compile
>> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently
>> triggered
>> >>>>>> by the addition of the procedure module on master)
>> >>>>>>
>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <palomino219@gmail.com>
wrote:
>> >>>>>>>
>> >>>>>>> Make findbugs and checkstyle plugins always run.
>> >>>>>>> The default behavior only publishes result for stable
builds, but
>> >>>>>>> master is
>> >>>>>>> not always stable and sometimes we introduce new warnings
in
>> unstable
>> >>>>>>> builds(see builds 6310-6314).
>> >>>>>>> And findbugs and checkstyle will not fail unless we
abort the
>> >>>>>>> building
>> >>>>>>> process I think, so it is safe to turn it on every time.
>> >>>>>>>
>> >>>>>>> Thanks.
>> >>>>>>>
>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
>> >>>>>>>
>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage
report.
>> >>>>>>> >
>> >>>>>>> > Cheers
>> >>>>>>> >
>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <palomino219@gmail.com>
>> wrote:
>> >>>>>>> >
>> >>>>>>> > > Going to change the config of HBase-TRUNK
jenkins to show
>> >>>>>>> > > findbugs,
>> >>>>>>> > > checkstyle and jacoco coverage report.
>> >>>>>>> > > The config has been tested on
>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco
for nearly
>> 30
>> >>>>>>> > > times.
>> >>>>>>> > Not
>> >>>>>>> > > much different from HBase-TRUNK unless it
runs ~30% slower(the
>> >>>>>>> > > overhead
>> >>>>>>> > of
>> >>>>>>> > > collecting information for code coverage).
>> >>>>>>> > > Thanks.
>> >>>>>>> > >
>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurtell@apache.org
>> >:
>> >>>>>>> > >
>> >>>>>>> > > > +1, thanks a lot for improving our build
hygiene.
>> >>>>>>> > > >
>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis
Söztutar
>> >>>>>>> > > > <enis.soz@gmail.com>
>> >>>>>>> > > wrote:
>> >>>>>>> > > >
>> >>>>>>> > > > > Thanks Sean. This is great.
>> >>>>>>> > > > >
>> >>>>>>> > > > > Enis
>> >>>>>>> > > > >
>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM,
Sean Busbey
>> >>>>>>> > > > > <busbey@cloudera.com>
>> >>>>>>> > > > wrote:
>> >>>>>>> > > > >
>> >>>>>>> > > > > > FYI I just finished chasing
down the breakage for mvn
>> site
>> >>>>>>> > > > > > on all
>> >>>>>>> > > patch
>> >>>>>>> > > > > > builds.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > HBASE-13191 consolidates the
few places in the
>> test-patch
>> >>>>>>> > > > > > code
>> >>>>>>> > where
>> >>>>>>> > > we
>> >>>>>>> > > > > > hard coded MAVEN_OPTS.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > If you look at the PreCommit
job now, we use the "set
>> >>>>>>> > > > > > environment
>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS
and then everything
>> >>>>>>> > > > > > else
>> >>>>>>> > respects
>> >>>>>>> > > > > that
>> >>>>>>> > > > > > setting.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > I set the initial value to
be a combination of the
>> memory
>> >>>>>>> > limitations
>> >>>>>>> > > > > we've
>> >>>>>>> > > > > > been actually running with
(the ~6G was getting ignored)
>> >>>>>>> > > > > > and the
>> >>>>>>> > > > permgen
>> >>>>>>> > > > > > needed for site.
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData
>> -XX:MaxPermSize=256m
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46
PM, Stack <
>> stack@duboce.net>
>> >>>>>>> > > > > > wrote:
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > > I just made TRUNK and
branch-1 builds use same jvm as
>> >>>>>>> > > > > > > patch-build
>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51
-- and I set the MAVEN_OPTS
>> to
>> >>>>>>> > > > > > > be the
>> >>>>>>> > > same
>> >>>>>>> > > > as
>> >>>>>>> > > > > > > those of trunk build too,
setting
>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"...
it
>> >>>>>>> > had
>> >>>>>>> > > > been
>> >>>>>>> > > > > > > 3000.
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > Yours,
>> >>>>>>> > > > > > > St.Ack
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at
4:22 PM, Stack <
>> stack@duboce.net>
>> >>>>>>> > > > > > > wrote:
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > > > > I upped hadoopqa
retention to keep last 100 builds
>> and
>> >>>>>>> > > > > > > > or last
>> >>>>>>> > 7
>> >>>>>>> > > > > days,
>> >>>>>>> > > > > > > > whichever comes first.
>> >>>>>>> > > > > > > > St.Ack
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014
at 9:38 AM, Stack
>> >>>>>>> > > > > > > > <stack@duboce.net>
>> >>>>>>> > wrote:
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > > >> Branch-1 and
master have stabilized and now run
>> mostly
>> >>>>>>> > > > > > > >> blue
>> >>>>>>> > > (give
>> >>>>>>> > > > or
>> >>>>>>> > > > > > > take
>> >>>>>>> > > > > > > >> the odd failure)
[1][2]. Having a mostly blue
>> branch-1
>> >>>>>>> > > > > > > >> has
>> >>>>>>> > > helped
>> >>>>>>> > > > us
>> >>>>>>> > > > > > > >> identify at least
one destabilizing commit in the
>> last
>> >>>>>>> > > > > > > >> few
>> >>>>>>> > days,
>> >>>>>>> > > > > maybe
>> >>>>>>> > > > > > > two;
>> >>>>>>> > > > > > > >> this is as it
should be (smile).
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> Lets keep our
builds blue. If you commit a patch,
>> make
>> >>>>>>> > > > > > > >> sure
>> >>>>>>> > > > > subsequent
>> >>>>>>> > > > > > > >> builds stay blue.
You can subscribe to
>> >>>>>>> > builds@hbase.apache.org
>> >>>>>>> > > to
>> >>>>>>> > > > > get
>> >>>>>>> > > > > > > >> notice of failures
if not already subscribed.
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> Thanks,
>> >>>>>>> > > > > > > >> St.Ack
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> 1.
>> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/
>> >>>>>>> > > > > > > >> 2.
>> >>>>>>> > >
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >> On Mon, Oct 13,
2014 at 4:41 PM, Stack
>> >>>>>>> > > > > > > >> <stack@duboce.net>
>> >>>>>>> > > wrote:
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >>> A few notes
on testing.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Too long
to read, infra is more capable now and
>> after
>> >>>>>>> > > > > > > >>> some
>> >>>>>>> > > work,
>> >>>>>>> > > > we
>> >>>>>>> > > > > > are
>> >>>>>>> > > > > > > >>> seeing branch-1
and trunk mostly running blue.
>> Lets
>> >>>>>>> > > > > > > >>> try and
>> >>>>>>> > > keep
>> >>>>>>> > > > it
>> >>>>>>> > > > > > > this
>> >>>>>>> > > > > > > >>> way going
forward.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Apache Infra
has new, more capable hardware.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> A recent
spurt of test fixing combined with more
>> >>>>>>> > > > > > > >>> capable
>> >>>>>>> > > hardware
>> >>>>>>> > > > > > seems
>> >>>>>>> > > > > > > >>> to have gotten
us to a new place; tests are mostly
>> >>>>>>> > > > > > > >>> passing
>> >>>>>>> > now
>> >>>>>>> > > on
>> >>>>>>> > > > > > > branch-1
>> >>>>>>> > > > > > > >>> and master.
 Lets try and keep it this way and
>> start
>> >>>>>>> > > > > > > >>> to trust
>> >>>>>>> > > our
>> >>>>>>> > > > > > test
>> >>>>>>> > > > > > > runs
>> >>>>>>> > > > > > > >>> again.  Just
a few flakies remain.  Lets try and
>> nail
>> >>>>>>> > > > > > > >>> them.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Our tests
now run in parallel with other test
>> suites
>> >>>>>>> > > > > > > >>> where
>> >>>>>>> > > > previous
>> >>>>>>> > > > > > we
>> >>>>>>> > > > > > > >>> ran alone.
You can see this sometimes when our
>> zombie
>> >>>>>>> > detector
>> >>>>>>> > > > > > reports
>> >>>>>>> > > > > > > >>> tests from
another project altogether as lingerers
>> >>>>>>> > > > > > > >>> (To be
>> >>>>>>> > > fixed).
>> >>>>>>> > > > > > > Some of
>> >>>>>>> > > > > > > >>> our tests
are failing because a concurrent hbase
>> run
>> >>>>>>> > > > > > > >>> is
>> >>>>>>> > undoing
>> >>>>>>> > > > > > > classes and
>> >>>>>>> > > > > > > >>> data from
under it. Also, lets fix.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Our tests
are brittle. It takes 75minutes for
>> them to
>> >>>>>>> > complete.
>> >>>>>>> > > > > Many
>> >>>>>>> > > > > > > >>> are heavy-duty
integration tests starting up
>> multiple
>> >>>>>>> > clusters
>> >>>>>>> > > > and
>> >>>>>>> > > > > > > >>> mapreduce
all in the one JVM. It is a miracle they
>> >>>>>>> > > > > > > >>> pass at
>> >>>>>>> > all.
>> >>>>>>> > > > > > > Usually
>> >>>>>>> > > > > > > >>> integration
tests have been cast as unit tests
>> >>>>>>> > > > > > > >>> because there
>> >>>>>>> > > was
>> >>>>>>> > > > no
>> >>>>>>> > > > > > > where
>> >>>>>>> > > > > > > >>> else for
them to get an airing.  We have the
>> hbase-it
>> >>>>>>> > > > > > > >>> suite
>> >>>>>>> > now
>> >>>>>>> > > > > which
>> >>>>>>> > > > > > > would
>> >>>>>>> > > > > > > >>> be a more
apt place but until these are run on a
>> >>>>>>> > > > > > > >>> regular
>> >>>>>>> > basis
>> >>>>>>> > > in
>> >>>>>>> > > > > > > public
>> >>>>>>> > > > > > > >>> for all to
see, the fat integration tests
>> disguised
>> >>>>>>> > > > > > > >>> as unit
>> >>>>>>> > > tests
>> >>>>>>> > > > > > will
>> >>>>>>> > > > > > > >>> remain. 
A review of our current unit tests
>> weeding
>> >>>>>>> > > > > > > >>> the old
>> >>>>>>> > > cruft
>> >>>>>>> > > > > and
>> >>>>>>> > > > > > > the
>> >>>>>>> > > > > > > >>> no longer
relevant or duplicates would be a nice
>> >>>>>>> > > > > > > >>> undertaking
>> >>>>>>> > if
>> >>>>>>> > > > > > > someone is
>> >>>>>>> > > > > > > >>> looking to
contribute.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> Alex Newman
has been working on making our tests
>> work
>> >>>>>>> > > > > > > >>> up on
>> >>>>>>> > > > travis
>> >>>>>>> > > > > > and
>> >>>>>>> > > > > > > >>> circle-ci.
 That'll be sweet when it goes
>> end-to-end.
>> >>>>>>> > > > > > > >>> He
>> >>>>>>> > also
>> >>>>>>> > > > > added
>> >>>>>>> > > > > > in
>> >>>>>>> > > > > > > >>> some "type"
categorizations -- client, filter,
>> >>>>>>> > > > > > > >>> mapreduce
--
>> >>>>>>> > > > > alongside
>> >>>>>>> > > > > > > our
>> >>>>>>> > > > > > > >>> old "sizing"
categorizations of
>> small/medium/large.
>> >>>>>>> > > > > > > >>> His
>> >>>>>>> > > thinking
>> >>>>>>> > > > > is
>> >>>>>>> > > > > > > that
>> >>>>>>> > > > > > > >>> we can run
these categorizations in parallel so we
>> >>>>>>> > > > > > > >>> could run
>> >>>>>>> > > the
>> >>>>>>> > > > > > total
>> >>>>>>> > > > > > > >>> suite in
about the time of the longest test, say
>> >>>>>>> > 20-30minutes?
>> >>>>>>> > > > We
>> >>>>>>> > > > > > > could
>> >>>>>>> > > > > > > >>> even change
Apache to run them this way.
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>> FYI,
>> >>>>>>> > > > > > > >>> St.Ack
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>>
>> >>>>>>> > > > > > > >>
>> >>>>>>> > > > > > > >
>> >>>>>>> > > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > >
>> >>>>>>> > > > > > --
>> >>>>>>> > > > > > Sean
>> >>>>>>> > > > > >
>> >>>>>>> > > > >
>> >>>>>>> > > >
>> >>>>>>> > > >
>> >>>>>>> > > >
>> >>>>>>> > > > --
>> >>>>>>> > > > Best regards,
>> >>>>>>> > > >
>> >>>>>>> > > >    - Andy
>> >>>>>>> > > >
>> >>>>>>> > > > Problems worthy of attack prove their
worth by hitting
>> back. -
>> >>>>>>> > > > Piet
>> >>>>>>> > Hein
>> >>>>>>> > > > (via Tom White)
>> >>>>>>> > > >
>> >>>>>>> > >
>> >>>>>>> >
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Sean
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sean
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Sean
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Sean
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Sean
>> >
>> >
>> >
>> >
>> > --
>> > Sean
>>
>>
>>
>> --
>> Sean
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message