hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nigel Daley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5069) add a Hadoop-centric junit test result listener
Date Wed, 28 Jan 2009 06:53:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667946#action_12667946

Nigel Daley commented on HADOOP-5069:

Steve, I like option 1 best as it seems simplest and achieves the goal of not buffering. 
But will option 1 work correctly? Aren't stdout and stderr separate elements in the XML? 
How can they be streamed to the file?

> add a Hadoop-centric junit test result listener
> -----------------------------------------------
>                 Key: HADOOP-5069
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5069
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Steve Loughran
>            Priority: Minor
> People are encountering different problems with hadoop's unit tests, defects currently
being WONTFIX'd
> # HADOOP-5001 : Junit tests that time out don't write any test progress related logs
> # HADOOP-4721 : OOM in .TestSetupAndCleanupFailure
> There is a root cause here, the XmlResultFormatter of Ant buffers everything before writing
out a DOM. Too much logged: OOM and no output. Timeout: kill and no output.
> We could add a new logger class to hadoop and then push it back into Ant once we were
happy, or keep it separate if we had specific dependencies (like on hadoop-dfs API) that they
> Some ideas
> # stream XML to disk. We would have to put the test summary at the end; could use XSL
to generate HTML and the classic XML content
> # stream XHTML to disk. Makes it readable as you go along; makes the XSL work afterwards
> # push out results as records to a DFS. There's a problem here in that this needs to
be a different DFS from that you are testing, yet it needs to be compatible with the client.

> Item #3 would be interesting but doing it inside JUnit is too dangerous classpath and
config wise. Better to have Ant do the copy afterwards. What is needed then is a way to easily
append different tests to the same DFS file in a way that tools can analyse them all afterwards.
The copy is easy -add a new Ant resource for that- but the choice of format is trickier.
> Here's some work I did on this a couple of years back; I've not done much since then:
> Is anyone else interested in exploring this? 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message