hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roman Shaposhnik (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4821) A fully automated comprehensive distributed integration test for HBase
Date Wed, 25 Apr 2012 18:20:23 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261885#comment-13261885
] 

Roman Shaposhnik commented on HBASE-4821:
-----------------------------------------

@Enis,

I had a really nice chat with Jon yesterday and we arrived at a common understanding that
the tests in general fall into 3 distinct categories (please note that this categories classify
test *implementation* and not whether they are used as part of the mvn test, mvn verify or
Bigtop's test infra -- more on that later): 
  # pure unit tests -- they reach into the guts of the implementation and use non-public APIs.
There's absolutely no way to run that testcode on anything but MiniHBase/MiniDFS/MiniMR. Bigtop
has no role to play in helping HBase community with developing/maintaining/executing those
tests.
  # HBase-specific functional tests -- these are the tests that only use public APIs and don't
muck about with internals. They are, however, only concerned with HBase itself. IOW, a test
that wants to verify that you can submit an Oozie workflow that has Hive->HBASE->Pig
pipeline does not fall into this category
  # Integration tests -- these are the multi-component tests that exercise  not just HBase
but a # of different components. An above example of the Oozie workflow falls into this category.

Here's how an ideal situation looks from Bigtop's perspective: 
  * you guys totally take care of #1 and you implement it as usual unit tests. 
  * Bigtop (with your help) takes care of #3. It simply makes no sense to reproduce the same
infra at the HBase level.
  * A proposal on #2 is this -- these tests belong to HBase. However, they have to be clearly
marked as belonging to the functional class AND they have to utilize a very thin shim layer
so you can use them in an mvn verify context and we can reuse them in Bigtop running against
a fully distributed beefy clusters. At this point I'm convinced that TestLoadAndVerify should
be the first example of this class and it should reside in HBase codebase (yet be available
to Bigtop).

Let me know if this makes sense.
                
> A fully automated comprehensive distributed integration test for HBase
> ----------------------------------------------------------------------
>
>                 Key: HBASE-4821
>                 URL: https://issues.apache.org/jira/browse/HBASE-4821
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>            Priority: Critical
>
> To properly verify that a particular version of HBase is good for production deployment
we need a better way to do real cluster testing after incremental changes. Running unit tests
is good, but we also need to deploy HBase to a cluster, run integration tests, load tests,
Thrift server tests, kill some region servers, kill the master, and produce a report. All
of this needs to happen in 20-30 minutes with minimal manual intervention. I think this way
we can combine agile development with high stability of the codebase. I am envisioning a high-level
framework written in a scripting language (e.g. Python) that would abstract external operations
such as "deploy to test cluster", "kill a particular server", "run load test A", "run load
test B" (we already have a few kinds of load tests implemented in Java, and we could write
a Thrift load test in Python). This tool should also produce intermediate output, allowing
to catch problems early and restart the test.
> No implementation has yet been done. Any ideas or suggestions are welcome.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message