hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: I wonder if it makes sense to restrict the HBase build to one build per jenkins host
Date Fri, 29 Aug 2014 23:20:34 GMT
I filed https://issues.apache.org/jira/browse/BUILDS-15


On Fri, Aug 29, 2014 at 4:13 PM, Stack <stack@duboce.net> wrote:

> Giri:
>
> Any chance of a heads up when you change the jenkins environment.  Only
> for Alex digging, we'd not have figured that the new fails were because we
> were running alongside another process.  We should make it so hbase tests
> are immune to concurrent test runs for sure we didn't know this was the
> prob.
>
> Via our Esteban, this seems to be showing we are running 4 executors
> https://builds.apache.org/computer/H11/load-statistics  Is that so?
>
> Let me file a builds issue with the above question to raise the profile.
> Thanks,
> St.Ack
>
>
> On Wed, Aug 27, 2014 at 6:31 PM, Giridharan Kesavan <
> gkesavan@hortonworks.com> wrote:
>
>> I recently added more than one executors per slave to get past the build
>> queueing along. Let me bring it down to one executors for now if thats
>> causing issues.
>>
>>
>> BTW, there is also a discussion about using dockers for build slaves.
>>
>>
>>
>> -giri
>>
>>
>> On Wed, Aug 27, 2014 at 6:06 PM, Enis Söztutar <enis@apache.org> wrote:
>>
>> > Yes,
>> >
>> > Giri still does some maintenance on those. He was saying that we will
>> > receive some more nodes from rackspace. Let me add him to the thread.
>> >
>> > Enis
>> >
>> >
>> > On Wed, Aug 27, 2014 at 1:59 PM, Nicolas Liochon <nkeywal@gmail.com>
>> > wrote:
>> >
>> >> It used to be the case for precommit builds: there was a single worker.
>> >> It's much simpler this way.
>> >> It seems it has changed, likely less than a year ago. I see on
>> >> https://builds.apache.org/computer/H0/load-statistics that there are 2
>> >> workers.
>> >> We're can't do what we want on these machines, as they are used by
>> other
>> >> projects however. In the past, Giri was administrating these machines,
>> I
>> >> don't know if it's still the case (Enis? Devaraj? Stack? Do you guys
>> >> know?)
>> >>
>> >> I guess that the root issue is that hdfs & other builds are not
>> >> parallelized as HBase are, so they need multiple workers to really use
>> the
>> >> multiple cores of the build env...
>> >>
>> >> Nicolas
>> >>
>> >>
>> >> On Wed, Aug 27, 2014 at 10:45 PM, Andrew Purtell <apurtell@apache.org>
>> >> wrote:
>> >>
>> >> > +1, if it's possible on ASF Jenkins
>> >> >
>> >> > We also share with other projects. I've seen build reports
>> implicating
>> >> > Oozie tests for supposed HBase test zombies.
>> >> >
>> >> > When we had cloud based Jenkins running at Trend Micro and Intel we
>> >> > launched instances on demand as we needed workers, but never more
>> than
>> >> one
>> >> > worker per instance concurrently. Definitely helps to reduce failures
>> >> due
>> >> > to unexpected state from interactions between tests.
>> >> >
>> >> >
>> >> > On Wed, Aug 27, 2014 at 1:28 PM, Alex Newman <posix4e@gmail.com>
>> wrote:
>> >> >
>> >> > > I have noticed that HBase tests can cause a good deal of inherent
>> test
>> >> > > interference. We have had much better luck running one and only
one
>> >> > > set of the tests at a time. I notice that this currently is not
>> >> > > happening. I don't have any rights on jenkins, so perhaps someone
>> else
>> >> > > could give this a shot. I'd be glad to describe how do it. Another
>> >> > > option would be to run the build from within a container (We do
>> this
>> >> > > at WanDISCO). Finally I got the build running on travis ci.
>> Checkout
>> >> > > https://github.com/OhmData/hbase-public/tree/travis for an
>> example of
>> >> > > how to get this done.
>> >> > >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best regards,
>> >> >
>> >> >    - Andy
>> >> >
>> >> > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> >> > (via Tom White)
>> >> >
>> >>
>> >
>> >
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message