hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Segel <michael_se...@hotmail.com>
Subject Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
Date Fri, 16 Sep 2011 09:05:44 GMT
I think you need to get a little bit more information.
Reducers are expensive. 
When Thomas says that he is aggregating data, what exactly does he mean?
When dealing w HBase, you really don't want to use a reducer.

You may want to run two map jobs and it could be that just dumping the output via jdbc makes
the most sense.

We are starting to see a lot of questions where the OP isn't providing enough information
so that the recommendation could be wrong...


Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 16, 2011, at 2:22 AM, Sonal Goyal <sonalgoyal4@gmail.com> wrote:

> There is a DBOutputFormat class in the org.apache,hadoop.mapreduce.lib.db
> package, you could use that. Or you could write to the hdfs and then use
> something like HIHO[1] to export to the db. I have been working extensively
> in this area, you can write to me directly if you need any help.
> 
> 1. https://github.com/sonalgoyal/hiho
> 
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
> 
> <http://in.linkedin.com/in/sonalgoyal>
> 
> 
> 
> 
> 
> On Fri, Sep 16, 2011 at 10:55 AM, Steinmaurer Thomas <
> Thomas.Steinmaurer@scch.at> wrote:
> 
>> Hello,
>> 
>> 
>> 
>> writing a MR-Job to process HBase data and store aggregated data in
>> Oracle. How would you do that in a MR-job?
>> 
>> 
>> 
>> Currently, for test purposes we write the result into a HBase table
>> again by using a TableReducer. Is there something like a OracleReducer,
>> RelationalReducer, JDBCReducer or whatever? Or should one simply use
>> plan JDBC code in the reduce step?
>> 
>> 
>> 
>> Thanks!
>> 
>> 
>> 
>> Thomas
>> 
>> 
>> 
>> 

Mime
View raw message