hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tsuna <tsuna...@gmail.com>
Subject Re: HBase Performance
Date Tue, 19 Apr 2011 18:16:41 GMT
On Wed, Apr 6, 2011 at 2:39 PM, Jean-Daniel Cryans <jdcryans@apache.org> wrote:
> Look for how Facebook is using HBase for messages. Also look for how
> we have been using HBase at StumbleUpon for 2 years now and for both
> live and batch queries. Numbers are usually included in the decks.

In addition to this, one of our clusters also houses our time series
database (see http://opentsdb.net), and it stores many billions of
data points, writes about 250 million new data points per day (that's
between 2500 and 3500 writes per second, 24/7), and it's frequently
asked to read back hundreds of thousands of data points per second
(when someone opens a dashboard for instance).  Note that this makes
up only a small fraction of the workload on that cluster.

The writes are fast.  I looked at a few hundred requests on one of our
TSDs (the application server of OpenTSDB that talks to HBase) and
here's what I saw for writes:
Average: 22ms
Median: 19ms (±14ms)
99th percentile: 62ms

Note that the TSD batches multiple writes together, so the timings
above are for batches of edits.  Batches have the following sizes:
Average: 15KB
Median: 10KB (±21KB)
99th percentile: 126KB

Reads of similar sizes tend to be equally fast, especially if you're
able to achieve a good hit rate in the block cache.

The main gotcha with using HBase for user-facing latency-sensitive
applications is that right now HBase is pretty slow to split or move
regions around.  When this happens, data will become unavailable for
up to several seconds.  You need to make sure that your application
can handle such periods of unavailability every once in a while.  When
an HBase server fails, it might take several seconds, or sometimes
over a minute for HBase to bring back online the data that was served
by that server onto another server.  The recovery time depends on many
factors (how many regions you have, what size they have, how many logs
and what size they have, chiefly).

Having said that, there are strategies to minimize the time during
which data is unavailable, so HBase can only improve in this respect.

-- 
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Mime
View raw message