hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4712) Document rules for writing tests
Date Fri, 02 Dec 2011 09:13:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161498#comment-13161498

nkeywal commented on HBASE-4712:

bq. Is that one report each for the medium and large or one report for the two? Just need
a single word of clarification in there 

One report for the two. I will reformulate this part.

bq. Is this new behavior from what's in trunk currently? Because I'm pretty sure I can just
run mvn test -Dtest=myTest and get it to run them.
Yes, it would change from what we have in trunk today. This profile deactivates categories,
if not re they are still taken into account even if you specify "-Dtest=myTest", so if your
test is in largeTests it's not executed.

bq. I don't think we really want to have these options available, right? A lot of the large
tests depend on the fact that jvms are forked.
My bad, copy/paste error, I am going to fix it

bq. A simple example or two here with the annotations would make this super obvious. For example:
Ok, will add an example.

bq. Do we need to add a MiniRegion just for testing these situations?
It runs like that out of the box (the region can use the local system), see for example TestScanner.

bq. It would be cool if we could actually do that with the annotation. For instance: @LargeTest(modifications=false)
If we can't check automatically that the annotation "does not lie", we have the risk of having
to look at the source code to check if it's true. As well, we can have test with a default
cluster but that cannot run with other tests (if they try to timout the master for example),
so I believe we will have to be explicit (something like sharedCluster=true). Note as well
that a test can belong to multiple categories.

Thanks for the review!
> Document rules for writing tests
> --------------------------------
>                 Key: HBASE-4712
>                 URL: https://issues.apache.org/jira/browse/HBASE-4712
>             Project: HBase
>          Issue Type: Task
>          Components: test
>    Affects Versions: 0.92.0
>            Reporter: nkeywal
>            Assignee: nkeywal
>            Priority: Minor
> We saw that some tests could be improved. Documenting the general rules could help.
> Proposal:
> HBase tests are divided in three categories: small, medium and large, with corresponding
JUnit categories: SmallTest, MediumTest, LargeTest
> Small tests are executed in parallel in a shared JVM. They must last less than 15 seconds.
They must NOT use a cluster.
> Medium tests are executed in separate JVM. They must last less than 50 seconds. They
can use a cluster. They must not fail occasionally.
> Small and medium tests must not need more than 30 minutes to run altogether.
> Small and medium tests should be executed by the developers before submitting a patch.
> Large tests are everything else. They are typically integration tests, non-regression
tests for specific bugs, timeout tests, performance tests.
> Tests rules & hints are:
> - As most as possible, tests should be written as small tests.
> - All tests should be written to support parallel execution on the same machine, hence
should not use shared resources as fixed ports or fixed file names.
> - All tests should be written to be as fast as possible.
> - Tests should not overlog. More than 100 lines/second makes the logs complex to read
and use i/o that are hence not available for the other tests.
> - Tests can be written with HBaseTestingUtility . This class offers helper function to
create a temp directory and do the cleanup, or to start a cluster.
> - Sleeps:
>     - Tests should not do a 'Thread.sleep' without testing an ending condition. This
allows understanding what the test is waiting for. Moreover, the test will work whatever the
machine performances.
>     - Sleep should be minimal to be as fast as possible. Waiting for a variable should
be done in a 40ms sleep loop. Waiting for a socket operation should be done in a 200 ms sleep
> - Tests using cluster:
>     - Tests using a HRegion do not have to start a cluster: A region can use the local
file system.
>     - Start/stopping a cluster cost around 10 seconds. They should not be started per
test method but per class.
>     - Started cluster must be shutdown using HBaseTestingUtility#shutdownMiniCluster,
which cleans the directories.
>     - As most as possible, tests should use the default settings for the cluster. When
they don't, they should document it. This will allow to share the cluster later.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message