hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Data taking up too much space when put into HBase
Date Wed, 10 Nov 2010 05:46:20 GMT
Each value is stored with it's full key e.g. row key + family +
qualifier + timestamp + offsets. You don't give any information
regarding how you stored the data, but if you have large enough keys
then it should easily explain the bloat.

J-D

On Tue, Nov 9, 2010 at 9:21 PM, Hari Sreekumar <hsreekumar@clickable.com> wrote:
> Hi,
>
>     Data seems to be taking up too much space when I put into HBase. e.g, I
> have a 2 GB text file which seems to be taking up ~70 GB when I dump into
> HBase. I have block size set to 64 MB and replication=3, which I think is
> the possible reason for this expansion. But if that is the case, how can I
> prevent it? Decreasing the block size will have a negative impact on
> performance, so is there a way I can increase the average size on
> HBase-created  files to be comparable to 64 MB. Right now they are ~5 MB on
> average. Or is this an entirely different thing at work here?
>
> thanks,
> hari
>

Mime
View raw message