hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Bautin (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4821) A fully automated comprehensive distributed integration test for HBase
Date Sat, 19 Nov 2011 01:40:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153319#comment-13153319
] 

Mikhail Bautin commented on HBASE-4821:
---------------------------------------

Thanks everyone for your comments. I will read up on BigTop and Puppet. I am also fine with
using a JVM-based language for load tests themselves, as long as there is a way to do something
like "kill -9", which we can't really do in our unit tests. We could also try to reuse/modify
the MiniHBaseCluster framework to talk to a real HBase cluster and script various distributed
test scenarios in pure Java.

However, I want to emphasize one thing. Once configured, this HBase integration test tool
should be extremely easy to use, as simple as: hbase_integration_test.sh <hbase_source_dir>.
We might have to write some amount of nontrivial "glue" script code (e.g. in bash) to make
that happen.
                
> A fully automated comprehensive distributed integration test for HBase
> ----------------------------------------------------------------------
>
>                 Key: HBASE-4821
>                 URL: https://issues.apache.org/jira/browse/HBASE-4821
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>            Priority: Critical
>
> To properly verify that a particular version of HBase is good for production deployment
we need a better way to do real cluster testing after incremental changes. Running unit tests
is good, but we also need to deploy HBase to a cluster, run integration tests, load tests,
Thrift server tests, kill some region servers, kill the master, and produce a report. All
of this needs to happen in 20-30 minutes with minimal manual intervention. I think this way
we can combine agile development with high stability of the codebase. I am envisioning a high-level
framework written in a scripting language (e.g. Python) that would abstract external operations
such as "deploy to test cluster", "kill a particular server", "run load test A", "run load
test B" (we already have a few kinds of load tests implemented in Java, and we could write
a Thrift load test in Python). This tool should also produce intermediate output, allowing
to catch problems early and restart the test.
> No implementation has yet been done. Any ideas or suggestions are welcome.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message