ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Gura <ag...@apache.org>
Subject Re: Loading Hbase data into Ignite
Date Tue, 11 Oct 2016 15:17:37 GMT

HBase regions doesn't map to Ignite nodes due to architectural differences.
Each HBase region contains rows in some range of keys that sorted
lexicographically while distribution of keys in Ignite depends on affinity
function and key hash code. Also how do you remap region to nodes in case
of region was splitted?

Of course you can get node ID in cluster for given key but because HBase
keeps rows sorted by keys lexicographically you should perform full scan in
HBase table. So the simplest way for parallelization data loading from
HBase to Ignite it concurrently scan regions and stream all rows to one or
more DataStreamer.

On Tue, Oct 11, 2016 at 4:11 PM, Anil <anilklce@gmail.com> wrote:

> HI,
> we have around 18 M records in hbase which needs to be loaded into ignite
> cluster.
> i was looking at
> http://apacheignite.gridgain.org/v1.7/docs/data-loading
> https://github.com/apache/ignite/tree/master/examples
> is there any approach where each ignite node loads the data of one hbase
> region ?
> Do you have any recommendations ?
> Thanks.

View raw message