hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irfan Mohammed <irfan...@gmail.com>
Subject ChainMapper/ChainReducer for hbase Tables
Date Tue, 23 Jun 2009 03:27:55 GMT
Don't know if this is a M/R question or hbase question but need help in solving the following
problem.

I have records with some dimensions and some metrics. In this case, lets say dimension DD
and the metrics [ M1- M2 ].

DD - M1 - M2
------------
------------
D1 - N1 - N2
D2 - N1 - N2

For a given record, I want to store each of the metrics into its own hbase table.

T_M1
----
     DD:D1 DD:D2
R1 -  N1    N1

T_M2
----
     DD:D1 DD:D2
R1 -  N2    N2

Right now, I have a mapper which converts the plain record into <ImmutableBytesWritable,
HbaseMapWritable<byte[], byte[]>> and the reducer puts into the Hbase table using
the <ImmutableBytesWritable, Put>. But the process starts all over again for the same
record for putting the data into the second metrics table. 

I want to be able to use ChainMapper/ChainReducer but was not successful. ChainReducer allows
only one reducer [setReducer] and there can be mappers after the reducer. attached is the
example code where in the test case, I am looping over the whole map/reduce for each table.


Is there a way to be able to do the following?

1. map
2. reduce --> t_m1
3. reduce --> t_m2

Thanks,
Irfan
Mime
View raw message