hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dima Spivak (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
Date Tue, 09 Aug 2016 01:17:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412788#comment-15412788

Dima Spivak commented on HBASE-12721:

Assuming you're running Docker 1.11+, to build HBase images from Git SHA (defaults to using
Hadoop 2.7.2):
source /dev/stdin <<< "$(curl -sL http://tiny.cloudera.com/clusterdock.sh)"
CLUSTERDOCK_TOPOLOGY_IMAGE=dimaspivak/clusterdock:apache_hbase_topology clusterdock_run ./bin/build_cluster
--namespace=dimaspivak apache_hbase --hadoop-version=2.7.1 --hadoop-tarball=https://archive.apache.org/dist/hadoop/core/hadoop-2.7.1/hadoop-2.7.1.tar.gz
--hbase-version=andysCommit --hbase-git-commit 1ecb0fce342ee878cf96f7a3165007192bedb2ef

To start the cluster:
CLUSTERDOCK_TOPOLOGY_IMAGE=dimaspivak/clusterdock:apache_hbase_topology clusterdock_run ./bin/start_cluster
--namespace=dimaspivak apache_hbase --hadoop-version=2.7.1 --hbase-version=andysCommit --secondary-nodes='node-{2..5}'

To run ITBLL against the cluster:
clusterdock_ssh node-1.cluster org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList...

> Create Docker container cluster infrastructure to enable better testing
> -----------------------------------------------------------------------
>                 Key: HBASE-12721
>                 URL: https://issues.apache.org/jira/browse/HBASE-12721
>             Project: HBase
>          Issue Type: New Feature
>          Components: build, community, documentation, test
>            Reporter: Dima Spivak
>            Assignee: Dima Spivak
> Some simple work on using HBase with Docker was committed into /dev-support as "hbase_docker;"
all this did was stand up a standalone cluster from source and start a shell. Now seems like
a good time to extend this to be useful for applications that could actual benefit the community,
especially around testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up distributed
HBase clusters on a single host machine in a couple minutes and run our awesome hbase-it suite
against it.
> - Binary compatibility testing of an HBase client is easiest when standing up an HBase
cluster can be done once and then different client source/binary permutations run against
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any upstream
automation on build.apache.org, in part because it's a pain to set up x-node clusters on Apache
> This proposal, whether it stays under /dev-support or moves out into it's own top-level
module ("hbase-docker" would conveniently fit the existing schema :-)), strives to create
a simple framework for deploying "distributed," multi-container Apache HBase clusters.

This message was sent by Atlassian JIRA

View raw message