hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7298) Add test utility for writing multi-threaded tests
Date Tue, 26 Jul 2011 18:27:09 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071261#comment-13071261
] 

Joep Rottinghuis commented on HADOOP-7298:
------------------------------------------

Todd, in the waitFor method why do you keep a count of finished and then wait in a loop?
Wouldn't a CountDownLatch work at least as well?

A problem I'm seeing in many tests is that they sleep for some # seconds in a loop.
For one single test that does not seem to matter too much, but adding it all up that would
result in a lot of unnecessary waiting. If one waits x seconds in a loop, then on average
1/2 x of time is wasted. The total quite of tests runs in about 4 hours (on 0.20-security)
even when I throw bigger hardware at it.
When a CountDownLatch is used to coordinate between threads (either to signal that work can
start, or that threads have finished) then there is no such additional waiting.

> Add test utility for writing multi-threaded tests
> -------------------------------------------------
>
>                 Key: HADOOP-7298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7298
>             Project: Hadoop Common
>          Issue Type: Test
>          Components: test
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7298.txt, hadoop-7298.txt, hadoop-7298.txt
>
>
> A lot of our tests spawn off multiple threads in order to check various synchronization
issues, etc. It's often tedious to write these kinds of tests because you have to manually
propagate exceptions back to the main thread, etc.
> In HBase we have developed a testing utility which makes writing these kinds of tests
much easier. I'd like to copy that utility into Hadoop so we can use it here as well.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message