Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
Date: Thu, 5 May 2011 15:26:43 -0600
Message-ID: <BANLkTikdfb=bUdiGsv50GxPveLAdbjf1YA@mail.gmail.com>
Subject: Considerations for using HBase in User Facing applications
From: Matt Davies <matt@mattdavies.net>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=20cf303bff4a9afe6604a28e07a7

--20cf303bff4a9afe6604a28e07a7
Content-Type: text/plain; charset=ISO-8859-1

Afternoon everyone,

I am researching what the best practice is for using HBase in user facing
applications.  I do not know all of the applications that will be ported to
use HBase, but they do share common characteristics such as

- simple key / value data.  Not serving large files ATM. Perhaps a couple
columns in a single column family
- very tall tables
- hundreds of millions of rows
- need millisecond access times for a single row
- random access
- maintain very, very good query times while loading in new data


The quick choice would be to use something like memcache or Redis, but the
data is growing faster than the memory of a single box or even few boxes.
 We also have a significant investment in Hadoop technologies so keeping
HBase prime seems to make a lot of sense.

So, some questions:

1. do you find that having a single HBase cluster to serve all applications
vs smaller clusters to serve application specific data is better?
2. In the real world do people hook API's directly to HBase or is there some
caching layer that is used?
3. I remember hearing people like StumbleUpon use different clusters for
analytics vs customer apps.  Is this still best practice?
4. Anyone using MSLAB's to reduce GC pauses in production? Experiences /
landmines?
5. What other considerations have you found when hooking HBase up for
user-facing applications?

Thanks in advance and I'd love to hear some bragging!

-Matt

--20cf303bff4a9afe6604a28e07a7--