hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: no memory tables
Date Fri, 27 Mar 2009 08:14:48 GMT

I'm really looking forward to taking HFile for a spin. Thanks so
much for your contributions, Ryan. 

  - Andy

> From: Ryan Rawson <ryanobjc@gmail.com>
> Subject: Re: no memory tables
> To: hbase-user@hadoop.apache.org
> Date: Thursday, March 26, 2009, 11:31 PM
> Hey,
> 
> Interesting ideas - there are some features in 0.20 that
> might obviate the need for some of the suggestions below...
> 
> One major problem with hbase 0.19 is the indexing scheme -
> an index entry is created every 128 entries.  With large
> data sets with small key/values, this is a major problem.
> 
> But in hbase 0.20, the index is now based on blocks.  On my
> own test:
> - 1 hfile that is 161 MB on disk
> - contains 11m key/values
> - represents about 5.5 million rows
> - 3.7x compression
> - default block size (pre-compression) of 64kBytes
> - in-memory block index size: 770kBytes.
> 
> One problem with 0.19 is the size of in-memory indexes...
> With hfile in 0.20 we will have many less problems.



      

Mime
View raw message