hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giridharan Kesavan <gkesa...@hortonworks.com>
Subject Re: I wonder if it makes sense to restrict the HBase build to one build per jenkins host
Date Thu, 28 Aug 2014 01:31:50 GMT
I recently added more than one executors per slave to get past the build
queueing along. Let me bring it down to one executors for now if thats
causing issues.


BTW, there is also a discussion about using dockers for build slaves.



-giri


On Wed, Aug 27, 2014 at 6:06 PM, Enis Söztutar <enis@apache.org> wrote:

> Yes,
>
> Giri still does some maintenance on those. He was saying that we will
> receive some more nodes from rackspace. Let me add him to the thread.
>
> Enis
>
>
> On Wed, Aug 27, 2014 at 1:59 PM, Nicolas Liochon <nkeywal@gmail.com>
> wrote:
>
>> It used to be the case for precommit builds: there was a single worker.
>> It's much simpler this way.
>> It seems it has changed, likely less than a year ago. I see on
>> https://builds.apache.org/computer/H0/load-statistics that there are 2
>> workers.
>> We're can't do what we want on these machines, as they are used by other
>> projects however. In the past, Giri was administrating these machines, I
>> don't know if it's still the case (Enis? Devaraj? Stack? Do you guys
>> know?)
>>
>> I guess that the root issue is that hdfs & other builds are not
>> parallelized as HBase are, so they need multiple workers to really use the
>> multiple cores of the build env...
>>
>> Nicolas
>>
>>
>> On Wed, Aug 27, 2014 at 10:45 PM, Andrew Purtell <apurtell@apache.org>
>> wrote:
>>
>> > +1, if it's possible on ASF Jenkins
>> >
>> > We also share with other projects. I've seen build reports implicating
>> > Oozie tests for supposed HBase test zombies.
>> >
>> > When we had cloud based Jenkins running at Trend Micro and Intel we
>> > launched instances on demand as we needed workers, but never more than
>> one
>> > worker per instance concurrently. Definitely helps to reduce failures
>> due
>> > to unexpected state from interactions between tests.
>> >
>> >
>> > On Wed, Aug 27, 2014 at 1:28 PM, Alex Newman <posix4e@gmail.com> wrote:
>> >
>> > > I have noticed that HBase tests can cause a good deal of inherent test
>> > > interference. We have had much better luck running one and only one
>> > > set of the tests at a time. I notice that this currently is not
>> > > happening. I don't have any rights on jenkins, so perhaps someone else
>> > > could give this a shot. I'd be glad to describe how do it. Another
>> > > option would be to run the build from within a container (We do this
>> > > at WanDISCO). Finally I got the build running on travis ci.  Checkout
>> > > https://github.com/OhmData/hbase-public/tree/travis for an example of
>> > > how to get this done.
>> > >
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>> >
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message