accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Havanki" <bhava...@clouderagovt.com>
Subject Review Request 18317: ACCUMULO-2380 - More useful output for functional tests run under MR
Date Thu, 20 Feb 2014 19:16:53 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18317/
-----------------------------------------------------------

Review request for accumulo.


Bugs: ACCUMULO-2380
    https://issues.apache.org/jira/browse/ACCUMULO-2380


Repository: accumulo


Description
-------

Changes to the functional test framework for more useful output when run under MapReduce.

* run.py supports a new -m option. When passed, it produces an extra line of output per test
with a code for the test outcome and the test's name (string form of the testcase object).
The string is used as the output from the map phase of RunTests.
* run.py also exits with status code 1 if any test does not pass, to help with scripting.
* The clean_logging method in TestUtils.py was made forgiving when deleting temporary log
configuration files. The need for this was seen during testing.
* The mapper in RunTests, which runs each functional test, was updated to:
  * pass -m to run.py
  * only send the special output triggered by -m to the reduce phase
  * insert the Hadoop task attempt ID into the special output, to allow testers to trace a
test run to its map task
  * increment counters based on test outcome

The net result of these changes is that a hadoop fs -cat of the reducer output files from
the job becomes a report on the overall test results, with a line for each test run with its
task attempt ID, outcome code, and task name. In addition, counters can be observed during
and after the run to get a glimpse of overall status.


Diffs
-----

  src/server/src/main/java/org/apache/accumulo/server/test/functional/RunTests.java 1a19bb5799382d22bf1b7e7bffd22f9f201ccaf4

  test/system/auto/TestUtils.py f36b01794395976c12c0124078cb4aac46808f56 
  test/system/auto/run.py 447539866e4760682218c397cc63548112675032 

Diff: https://reviews.apache.org/r/18317/diff/


Testing
-------

Ran functional tests under MapReduce on a 3-node VM cluster. Verified the correctness of the
reducer output and use of counters. Also tested running run.py directly for correct output
generation with -m there.


Thanks,

Bill Havanki


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message