hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Dechoux <decho...@gmail.com>
Subject Re: Simple way to unit test HBase Map reduce jobs?
Date Tue, 25 Sep 2012 07:18:37 GMT
"Apache MRUnit ™ is a Java library that helps developers unit test Apache
Hadoop map reduce jobs."

That is every kind of mapper and reducer can be tested with it. That is to
say, there is no specific support of HBase.
If you can manage by yourself to provide the key/value that you should
received from HBase, MRUnit is the right tool for testing the logic that
should work on these data.
With MRUnit, you will can write fine grained unit tests.

There is currently no stable support of miniCluster. If there is a
mishandling of data (like using a shared static variable), it may not show
with MRunit.

I am curious about the 'miniserver'. That's an equivalent of miniCluster
but for hbase processes/daemons? Or was that your question? It could indeed
be very useful for local integration test.

Regards

Bertrand

On Tue, Sep 25, 2012 at 8:55 AM, Eugeny Morozov
<emorozov@griddynamics.com>wrote:

> Hi, Elazar,
>
> I've found that MRUnit is pretty convenient way to test MR jobs.
> On the other hand there is HBaseTestingUtility, which is helpful to run
> miniCluster.
>
> Hope this helps.
>
> On Mon, Sep 24, 2012 at 8:43 PM, Elazar Leibovich <elazarl@gmail.com>
> wrote:
>
> > Is there a way similar to miniserver to test map reduce jobs on something
> > similar to miniCluster?
> >
>
>
>
> --
> Evgeny Morozov
> Developer Grid Dynamics
> Skype: morozov.evgeny
> www.griddynamics.com
> emorozov@griddynamics.com
>



-- 
Bertrand Dechoux

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message