hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sriram Muthuswamy Chittathoor" <srir...@ivycomptech.com>
Subject RE: public numbers for IHBase? (was Re: Support for MultiGet / SQL In clause -- error in patch HBASE-1845)
Date Mon, 25 Jan 2010 19:17:46 GMT
What I am finding is that it really hogs memeory when I was trying to insert rows and the region
server kept crashing on me.  For example I was able to successfully create 2 million rows
with around 2 GB but after that it keep needing more memory.

Is this the experience with everyone or I am doing something wrong.  Basically at most on
my Linux box I can go upto 2.7 GB on a 32 b it JVM.  For 1 billion rows and using IHBase --
what kind of memeory do I need 

Thanks



-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, January 26, 2010 12:00 AM
To: hbase-user@hadoop.apache.org
Subject: Re: public numbers for IHBase? (was Re: Support for MultiGet / SQL In clause -- error
in patch HBASE-1845)

No real numbers at the moment.  HBASE-2167 adds a
PerformanceEvaluation for IHBase (Indexed HBase).  PE is sort of not
the right use-case for IHBase with its largish, random values -- the
latter requires RAM and writes are slowed.  Nonetheless, search for
random values with the IHBase index can be up to two orders of
magnitude better in this hostile test: e.g.  20 scans for 20 random
values on a single node cluster with 1.5GB of memory allocated to the
RS VM.

Without an index: 732989ms at offset 0 for 1048576 rows
With an index: 2160ms at offset 0 for 1048576 rows

St.Ack

On Sun, Jan 24, 2010 at 1:17 AM, Andrew Purtell <apurtell@apache.org> wrote:
> Stack, any way you might persuade the IHBase guys to post some numbers publicly?
> I'd like to know more.
>
>   - Andy
>
>
>
> ----- Original Message ----
>> From: Stack <stack@duboce.net>
>> Subject: Re: Support for MultiGet / SQL In clause -- error in patch HBASE-1845
> [...]
>> Let us know how IHBase works out for you (indexed hbase).  Its a RAM
>> hog but the speed improvement finding matching cells can be startling.
>
>
>
>
>

This email is sent for and on behalf of Ivy Comptech Private Limited. Ivy Comptech Private
Limited is a limited liability company.  

This email and any attachments are confidential, and may be legally privileged and protected
by copyright. If you are not the intended recipient dissemination or copying of this email
is prohibited. If you have received this in error, please notify the sender by replying by
email and then delete the email completely from your system. 
Any views or opinions are solely those of the sender.  This communication is not intended
to form a binding contract on behalf of Ivy Comptech Private Limited unless expressly indicated
to the contrary and properly authorised. Any actions taken on the basis of this email are
at the recipient's own risk.

Registered office:
Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara Hills, Hyderabad 500 033,
Andhra Pradesh, India. Registered number: 37994. Registered in India. A list of members' names
is available for inspection at the registered office.


Mime
View raw message