hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Yates (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4448) HBaseTestingUtilityFactory - pattern for re-using HBaseTestingUtility instances across unit tests
Date Sat, 01 Oct 2011 21:56:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118916#comment-13118916
] 

Jesse Yates commented on HBASE-4448:
------------------------------------

bq. we're not completely dependent on tests being thorough about cleanu

I don't think it would be completely unreasonable to expect tests to cleanup after themselves.
It can be a bit of a pain but reseting static variables, etc is good practice. Thats why the
cleanup method is there. If we do go through all the tests and make sure they don't leave
a dirty jvm, we could switch to running tests in 'parallel' mode, rather than forked. This
gets us faster running tests - you don't teardown and create a new jvm each time and things
will more easily actually run in parallel (I think on the backend it just fires up a bunch
of threads in the same jvm and starts ticking off tests). It also means we get a more meaningful
output from maven - not just the number of fails, but also which classes failed and the stack
traces (rather than having to grep through for <<<FAILURE); its a minor inconvenience
but nice to have.

bq. Classes and NOT in separate jvms. I couldn't make it work. Seems like surefire-plugin
is a little buggy doing tests in parallel; its a newish feature.

I don't think that maven is really good about parallelizing wrt to forked tests, but I _think_
we can also parallelize the work among several build servers (though I'm not familiar with
the Apache build system, so we many not have multiple servers to distribute work to). 

bq. Can we work on aggregating our tests more so that more tests get run everytime we spin
up a cluster

I just want to make sure we don't start slamming things together just so we cut down on creating
clusters, but that their functionality actually belongs together. I would rather a slower
build and more confidence in the test coverage.


                
> HBaseTestingUtilityFactory - pattern for re-using HBaseTestingUtility instances across
unit tests
> -------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-4448
>                 URL: https://issues.apache.org/jira/browse/HBASE-4448
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Doug Meil
>            Assignee: Doug Meil
>            Priority: Minor
>         Attachments: HBaseTestingUtilityFactory.java, hbase_hbaseTestingUtility_uses_2011_09_22.xlsx,
java_HBASE_4448.patch, java_HBASE_4448_v2.patch
>
>
> Setting up and tearing down HBaseTestingUtility instances in unit tests is very expensive.
 On my MacBook it takes about 10 seconds to set up a MiniCluster, and 7 seconds to tear it
down.  When multiplied by the number of test classes that use this facility, that's a lot
of time in the build.
> This factory assumes that the JVM is being re-used across test classes in the build,
otherwise this pattern won't work. 
> I don't think this is appropriate for every use, but I think it can be applicable in
a great many cases - especially where developers just want a simple MiniCluster with 1 slave.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message