hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: import data from oracle to hbase: how much more space need to reserver?
Date Tue, 09 Aug 2011 16:44:46 GMT
Yes, all the keys are stored along all the values.

J-D

On Mon, Aug 8, 2011 at 11:50 PM, Steinmaurer Thomas
<Thomas.Steinmaurer@scch.at> wrote:
> Hello,
>
> this is an interesting link. If I get one thing right, is it really true
> that the row-key is stored per cell? If so, I guess we have to re-think
> our current row-key which is combined by three fields using 48 bytes
> total.
>
> Thanks,
> Thomas
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of
> Jean-Daniel Cryans
> Sent: Montag, 08. August 2011 20:26
> To: user@hbase.apache.org
> Subject: Re: import data from oracle to hbase: how much more space need
> to reserver?
>
> Using compression is important but so is designing the keys to be as
> small as possible. That means using a family name of 1 character if
> possible (we use "d" for "default" usually here), but also taking time
> to cleverly design your row keys. OpenTSDB is a good example:
> http://opentsdb.net/schema.html
>
> Shaving off 5 bytes of a key on 10B cells is about 46GB in savings
> pre-compression, but that also counts when storing that in caches,
> compacting, etc.
>
> J-D
>
> On Sun, Aug 7, 2011 at 8:35 PM, alex zhang <alexzhang2008@gmail.com>
> wrote:
>> it's depend on the orginal column number in oracle, and the data
>> length in the oracle.
>> the key in hbase include each row key+column name+timestape+TTL. so if
>
>> you have many column and the actually column value is very small, then
>
>> you the hbase table is much bigger then table in oracle.
>> So be sure use compress in hbase.
>>
>> I had a test table in oracle is 3G, after import to hbase, it's 28G.
>> with gz compress it about 4G, with lzo compress it's about 7G.
>>
>> Thanks
>> Zhang, Gui
>>
>> 2011/8/6 Daniel,Wu <hadoop_wu@163.com>
>>
>>> if a table in oracle has a size of 100T, and then put it into hbase,
>>> how much space normally will hbase take?
>>>
>>
>

Mime
View raw message