hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dima Spivak (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
Date Tue, 28 Jun 2016 01:29:57 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352173#comment-15352173
] 

Dima Spivak edited comment on HBASE-12721 at 6/28/16 1:29 AM:
--------------------------------------------------------------

Yeah, give it a shot. You can pass a comma-separated list of directories into {{\-\-data-directories}}
and it'll set up YARN and HDFS accordingly. As an example, my instances have SSDs mounted
into {{/data}}, {{/data1}}, {{/data2}}, etc. My invocation to start up nodes which we use
to run {{hbase-it}} nightly is:
{noformat}
clusterdock_run ./bin/start_cluster --always-pull -n network${RANDOM} apache_hbase \
    --hbase-version=${HBASE_BRANCH} --secondary-nodes='node-{2..5}' \
    --data-directories='/data,/data1,/data2,/data3,/data4,/data5' --start-services
{noformat}



was (Author: dimaspivak):
Yeah, give it a shot. You can pass a comma-separated list of directories into {{\-\-data-directories}}
and it'll set up YARN and HDFS accordingly. As an example, my instances have SSDs mounted
into {{/data}}, {{/data1}}, {{/data2}}, etc. My invocation to start up nodes which we use
to run {{hbase-it}} nightly is:
{noformat}
clusterdock_run ./bin/start_cluster --always-pull -n network${RANDOM} apache_hbase \
    --hbase-version=${HBASE_BRANCH} --secondary-nodes='node-{2..5}.internal' \
    --data-directories='/data,/data1,/data2,/data3,/data4,/data5' --start-services
{noformat}


> Create Docker container cluster infrastructure to enable better testing
> -----------------------------------------------------------------------
>
>                 Key: HBASE-12721
>                 URL: https://issues.apache.org/jira/browse/HBASE-12721
>             Project: HBase
>          Issue Type: New Feature
>          Components: build, community, documentation, test
>            Reporter: Dima Spivak
>            Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support as "hbase_docker;"
all this did was stand up a standalone cluster from source and start a shell. Now seems like
a good time to extend this to be useful for applications that could actual benefit the community,
especially around testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up distributed
HBase clusters on a single host machine in a couple minutes and run our awesome hbase-it suite
against it.
> - Binary compatibility testing of an HBase client is easiest when standing up an HBase
cluster can be done once and then different client source/binary permutations run against
it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any upstream
automation on build.apache.org, in part because it's a pain to set up x-node clusters on Apache
infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own top-level
module ("hbase-docker" would conveniently fit the existing schema :-)), strives to create
a simple framework for deploying "distributed," multi-container Apache HBase clusters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message