hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: import data from oracle to hbase: how much more space need to reserver?
Date Mon, 08 Aug 2011 18:26:09 GMT
Using compression is important but so is designing the keys to be as
small as possible. That means using a family name of 1 character if
possible (we use "d" for "default" usually here), but also taking time
to cleverly design your row keys. OpenTSDB is a good example:
http://opentsdb.net/schema.html

Shaving off 5 bytes of a key on 10B cells is about 46GB in savings
pre-compression, but that also counts when storing that in caches,
compacting, etc.

J-D

On Sun, Aug 7, 2011 at 8:35 PM, alex zhang <alexzhang2008@gmail.com> wrote:
> it's depend on the orginal column number in oracle, and the data length in
> the oracle.
> the key in hbase include each row key+column name+timestape+TTL. so if you
> have many column and the actually column value is very small, then you the
> hbase table is much bigger then table in oracle.
> So be sure use compress in hbase.
>
> I had a test table in oracle is 3G, after import to hbase, it's 28G.
> with gz compress it about 4G, with lzo compress it's about 7G.
>
> Thanks
> Zhang, Gui
>
> 2011/8/6 Daniel,Wu <hadoop_wu@163.com>
>
>> if a table in oracle has a size of 100T, and then put it into hbase, how
>> much space normally will hbase take?
>>
>

Mime
View raw message