hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vaibhav Puranik <vpura...@gmail.com>
Subject Scanner call crashed regionserver under load
Date Thu, 15 Jul 2010 03:56:48 GMT
Hi all,


We experienced a downtime in our HBase installation today.

We have our HBase hosted in EC2 with 1 master (with ZK) and 3 slaves (all of
them are m1.large). We are using HBase version 0.20.4

We have a method that opens a scanner and retrieves some values. The method
approximately scans 300 rows. Each row has three column families and
approximately 75 longs.
The table is fairly small and in total the table has approximately 1500
rows.

We tried calling this method under huge traffic and the CPU of the
regionserver that hosted this table spiked to 100%. It brought down our
application.

We are doing multiple incrementColumnValue calls for every request for this
traffic and HBase seems to take it well.

So, does that mean, it's a bad idea to call a scanner under huge traffic?
Will this problem get solved if we make a new table and store the values
with a different key so that the exact value can be retrieved (with a Get
call) ? Are there any other ways to resolve this hotspotting without
duplicating data?

Are we limited by what a machine can handle if we have a fairly small table
(that can fit in a region server or possibly in a single region)? Are there
any creative solutions people are using?

Regards,
Vaibhav Puranik
GumGum

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message