hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5621) MapReducer to run junit tests under Hadoop
Date Fri, 03 Apr 2009 21:10:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695580#action_12695580
] 

Steve Loughran commented on HADOOP-5621:
----------------------------------------

HADOOP-1527 is running Ant's <junit> task as the job, runs the tests in a different
VM, generating, text output which it then tries to parse in. It will need ant.jar, ant-optional.jar
and junit on the path.

This one does it in VM -which has a price -less robust against JVM crashes, and there isn't
any timeout enforcement logic there yet. Where the big difference is is that the output comes
out serialized, one test result is emitted for every method that passes/fails, with the exceptions
and stack trace included. This makes it a more powerful format for analysis.

The brittleness/timeout is a problem, but then its a problem in <junit> too. What could
be done would be to exec() a child process for doing the work, with it throwing back the serialized
test result, with the MR host process killing things if they took too long -but still saving
everything that got forwarded out. That would be a trickier undertaking, but possible. 

Also: I pass down a configuration to the test cases if they have the interface to receive
it. Right now, that is the same config that the MR job gets, but you could imagine more configuration-driven
work where every map line includes not just the test method but some config information, so
someone could do more configuration driven work. Of course, if you want to do that then the
test reports had better include the entire configuration, for better examination of what triggered
the failures.

It's currently built against junit3.8.2; moving to junit4 is worthwhile, but I'd have to use
a slightly different test runner

What '1257 has is an ant task that looks more like a drop in replacement for <junit>;
a specific launcher for the job. This proposals workflow would start off generating a list
of tests to run then pushing that out as part of a blocking job submission. 

> MapReducer to run junit tests under Hadoop
> ------------------------------------------
>
>                 Key: HADOOP-5621
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5621
>             Project: Hadoop Core
>          Issue Type: New Feature
>    Affects Versions: 0.21.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>
> This is something I mentioned to some people last week, thought I would start a discussion
on it.
> We could run junit tests as a MapReduce job with
> # a mapper that takes a list of classes, one per line
> # extracts the test suite from each class, and then invokes each test method. This would
be a new junit test runner.
> # saves the result (and any exceptions) as the output. Also saves any machine specific
details. 
> # It also needs to grab the System.out and System.err channels, to map them to specific
tests.
> # Measure how long the tests took (incuding setup/teardown time)
> # Add an ant task <listresources> to take filesets and other patterns, and generate
text files from the contents (with stripping of prefixes and suffices, directory separator
substition, file begin/end values, etc, etc). I have this with tests already.
> The result would be that you could point listresources at a directory tree and create
a text file listing all tests to run. These could be executed across multiple hosts and the
results correlated. It would be, initially, a MapExpand, as the output would be bigger than
the input
> Feature creep then becomes the analysis
> # Add another MR class which runs through all failing tests and creates a new list of
test classes that failed. This could be rescheduled on different runs, and makes for a faster
cycle (only run failing tests until they work)
> # Add something to only get failing tests, summarise them (somehow) in a user readable
form
> # Something to get partially failing tests and highlight machine differences. 
> # Add something to compare tests over time, detect those which are getting slower?
> # an MR to regenerate the classic Ant junit XML reports, for presentation in other tools
(like hudson)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message