hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Data taking up too much space when put into HBase
Date Wed, 10 Nov 2010 05:46:20 GMT
Each value is stored with it's full key e.g. row key + family +
qualifier + timestamp + offsets. You don't give any information
regarding how you stored the data, but if you have large enough keys
then it should easily explain the bloat.


On Tue, Nov 9, 2010 at 9:21 PM, Hari Sreekumar <hsreekumar@clickable.com> wrote:
> Hi,
>     Data seems to be taking up too much space when I put into HBase. e.g, I
> have a 2 GB text file which seems to be taking up ~70 GB when I dump into
> HBase. I have block size set to 64 MB and replication=3, which I think is
> the possible reason for this expansion. But if that is the case, how can I
> prevent it? Decreasing the block size will have a negative impact on
> performance, so is there a way I can increase the average size on
> HBase-created  files to be comparable to 64 MB. Right now they are ~5 MB on
> average. Or is this an entirely different thing at work here?
> thanks,
> hari

View raw message