hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: IntegrationTestBigLinkedList now running on builds.apache.org
Date Tue, 23 Aug 2016 02:57:47 GMT
On Mon, Aug 22, 2016 at 5:33 PM, Dima Spivak <dspivak@cloudera.com> wrote:

> Dear devs,
>
> tl;dr: We now have Jenkins jobs
> <https://builds.apache.org/job/HBase-master-IntegrationTestBigLinkedList/>
> that can run IntegrationTestBigLinkedList with fault injection on 5-node
> Apache HBase clusters built from source.
>
> Long version:
>
> I just wanted to provide an update on some recent work we've gotten done
> since committing an Apache HBase topology for clusterdock
> <https://github.com/apache/hbase/commit/ccf5d27d7aa238c8398d2818928a71
> f39bd749a0>
> (a Python-based framework for building and starting Docker container-based
> clusters).
>
> Despite the existence of an awesome system test framework with
> fault-injection capabilities in the form of the hbase-it module, we've
> never had an easy way to run these tests on distributed clusters upstream.
> This has long been a big hole in our Jenkins test coverage, but since the
> clusterdock topology got committed, we've been making progress on doing
> something about it. I'm happy to report that, starting today, we are now
> running IntegrationTestBigLinkedList with fault-injection on Apache
> Infrastructure
> <https://builds.apache.org/job/HBase-master-IntegrationTestBigLinkedList/>
> .
>
>
Hotdog!
St.Ack


> Even longer version (stop reading here if you don't care how we do it):
>
> So how do we do it? Well clusterdock is designed to start up multiple
> Docker containers on one host where each containers acts like a lightweight
> VM (so 4 containers = 4-node cluster). What's in these containers (and what
> to do when starting them) is controlled by clusterdock's "topology"
> abstraction. Our apache_hbase topology builds a Docker image from a Java
> tarball, Hadoop tarball, and an HBase version. This last part can be either
> a binary tarball (for RC testing or playing around with a release) or a Git
> commit, in which case our clusterdock topology builds HBase from source.
> Once we build a cluster, we can then push the cluster images (actually,
> just one Docker image) to a shared Docker registry for repeated use. We now
> have a matrix job that can build any branches we care about (I set it up
> against branch-1.2
> <https://builds.apache.org/view/H-L/view/HBase/job/HBase-
> Build-clusterdock-Clusters/HBASE_VERSION=branch-1.2,label=docker/>,
> branch-1.3
> <https://builds.apache.org/view/H-L/view/HBase/job/HBase-
> Build-clusterdock-Clusters/HBASE_VERSION=branch-1.3,label=docker/>,
> and master
> <https://builds.apache.org/view/H-L/view/HBase/job/HBase-
> Build-clusterdock-Clusters/HBASE_VERSION=master,label=docker/>
> to start) and do this.
>
> Once these images are built (and pushed), we can use them to start up an
> n-node sized cluster on one host and run tests against it. To begin, I've
> set up a super simple Jenkins job that starts up a 5-node cluster, runs
> ITBLL (with an optional Chaos Monkey), and then exits.
>
> This work is being tracked in HBASE-15964 and there's much more that I want
> to do (more tests, more Chaos Monkeys, more branches, more diagnostic
> information collection when a test fails), but I figured I'd let you guys
> know about what have going so far. :)
>
> PS: Special thanks to Jon Hsieh for helping me get the Jenkins jobs
> running.
>
> --
> -Dima
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message