Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of hsreekumar@clickable.com
 designates 74.125.149.77 as permitted sender)
MIME-Version: 1.0
Date: Sat, 26 Mar 2011 22:31:03 +0530
Message-ID: <AANLkTi=ofwyHO3qfWjs9mo+jxnCLjqwhq2RxEQ8Dw-Vi@mail.gmail.com>
Subject: HBase read latency
From: Hari Sreekumar <hsreekumar@clickable.com>
To: user <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=000e0cd56208da037e049f65a7a2

--000e0cd56208da037e049f65a7a2
Content-Type: text/plain; charset=ISO-8859-1

Hi guys,

On what factors does HBase read latency primarily depend? What would be the
approx theoretical limit for read latency in v0.90.1 on a cluster of 7 nodes
(16 core/16 GB RAM on 5 machines and 36 GB on the other two)? I have an
application where I generate around 1000 rows/s to be input into HBase. Then
I have to read this data and process it at regular intervals. Write speed is
not a problem as the cluster seems to be able to write at the reqd. rate.
But while processing this data also, I would need a read speed of at least
1000 rows/s since I need to keep the processing speed at least equal to the
data generation speed. So far, I am getting around 200-300 rows/s only it
seems. I have LZO compression on the tables and I haven't tried in-memory
yet as my RAM usage is too high already while running jobs. Is it possible
to achieve this read speed, and what can I do to improve it? How far can
adding more nodes/more RAM help? Please let me know if the scope is too huge
to answer this question and if you need more details.

Thanks,
Hari

--000e0cd56208da037e049f65a7a2--