hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lashing <lss...@gmail.com>
Subject Re: Bulkload or hbase API
Date Thu, 14 Mar 2013 16:51:01 GMT

Hi Hardy,
    Thanks for the tip. Will multiple (1000) concurrent Bulkload cause contention in hbase
? How will hbase handle so many hfile at the same time ? Thanks.

Damien Hardy <dhardy@viadeoteam.com> 於 2013/3/15 上午12:36 寫道:

> Hello Lashing,
> 
> MapReduce would be great :
> 
> Each mapper addresses a different MySQL DB and "TableOutputFormat" to the
> corresponding HTable.
> 
> maybe pig : UNION after LOAD on different MySQL DB and then STORE on the
> différent table according to your policy (may need several M/R jobs all
> managed by pig workflow).
> 
> The more efficient (1 job) would be pure home made Java MapReduce (mapper
> only for each MySQL DB bulk loading on HTables)
> 
> Cheers,
> 
> -- 
> Damien HARDY

Mime
View raw message