accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Valentin <ar...@arielvalentin.com>
Subject Testing Map Reduce Jobs
Date Wed, 29 Oct 2014 21:16:15 GMT
I am looking for some guidance that will help me write better tests for our
map reduce jobs. My current jobs are tested using MRUnit, which covers most
of the "logic" but I feel like I am missing good "end-to-end" developer
tests.

I took a look at the tests for mapred classes but I am not sure that it
achieves my goal of an end-to-end test because of the use of MockInstance.
https://github.com/apache/accumulo/blob/master/mapreduce/src/test/java/org/apache/accumulo/core/client/mapred{,reduce}/

For me the characteristic of a end-to-end test that I would find valuable
is a suite that one could execute using mini-{accumulo,yarn,et.al.} but I
don't see any examples of how one would go about making those components
work in concert with each other.

Does anyone have any guidance when it comes to writing automated developer
end-to-end tests?

What kinds of testing strategies are people out there using for MR jobs?

Thanks,
Ariel Valentin
e-mail: ariel@arielvalentin.com
website: http://blog.arielvalentin.com
skype: ariel.s.valentin
twitter: arielvalentin
linkedin: http://www.linkedin.com/profile/view?id=8996534
---------------------------------------
*simplicity *communication
*feedback *courage *respect

Mime
View raw message