hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4703) Improvements in tests
Date Sun, 30 Oct 2011 21:14:32 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139774#comment-13139774
] 

nkeywal commented on HBASE-4703:
--------------------------------

I don't have a generic solution. In some cases, it's possible to replace something like this:
{noformat}
for (i<1; i<nbTries; ++i){
  if (cond())
    break;
  Thread.sleep(1s);
}
{noformat}

by

{noformat}
while (!timeout(5s) && !cond()){
  Thread.sleep(0.1s);
}
{noformat}

On average the second solution takes 0.45s less than the first. Basically, in the tests we
should be more aggressive with a 3 nodes cluster than in production with a 100 nodes one.
I.e.: it makes sense to test a condition 10 times per second on the tests and 1 time per second
in prod. Ideally, there should be no sleep at all, just a wait for the right event. 

In tha patch, in most cases, the modifications are of the first type; just increasing the
number of tests per second, but not changing the timeout.

                
> Improvements in tests
> ---------------------
>
>                 Key: HBASE-4703
>                 URL: https://issues.apache.org/jira/browse/HBASE-4703
>             Project: HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.92.0
>         Environment: all
>            Reporter: nkeywal
>            Assignee: nkeywal
>            Priority: Minor
>         Attachments: 20111030_4703_all.patch, 20111030_4703_all.v2.patch
>
>
> Global:
>  - when possible, make the test using the default cluster configuration for the number
of region (1 instead of 2 or 3). This allows a faster stop/start, and is a step toward a shared
cluster configuration.
>  - 'sleep': lower or remove the sleep based synchronisation in the tests (in HBaseTestingUtility,
TestGlobalMemStoreSize, TestAdmin, TestCoprocessorEndpoint, TestHFileOutputFormat, TestLruBlockCache,
TestServerCustomProtocol, TestReplicationSink)
>   - Optimize 'put' by setting setWriteToWAL to false, when the 'put' is big or in a loop.
Not done for tests that rely on the WAL.
>  
> Local issues:
> - TestTableInputFormatScan fully deletes the hadoop.tmp.dir directory on tearDown, that
makes it impossible to use in // with another test using this directory
> - TestIdLock logs too much (9000 lines per seconds). Test time lowered to 15 seconds
to make it a part of the small subset
> - TestMemoryBoundedLogMessageBuffer useless System.out.println
> - io.hfile.TestReseekTo useless System.out.println
> - TestTableInputFormat does not shutdown the cluster
> - testGlobalMemStore does not shutdown the cluster
> - rest.client.TestRemoteAdmin: simplified, does not use local admin, single test instead
of two.
> - HBaseTestingUtility#ensureSomeRegionServersAvailable starts only one server, should
start the number of missing server instead.
> - TestMergeTool should starts/stops the dfs cluster with HBaseTestingUtility

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message