hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Sreekumar <hsreeku...@clickable.com>
Subject HBase read latency
Date Sat, 26 Mar 2011 17:01:03 GMT
Hi guys,

On what factors does HBase read latency primarily depend? What would be the
approx theoretical limit for read latency in v0.90.1 on a cluster of 7 nodes
(16 core/16 GB RAM on 5 machines and 36 GB on the other two)? I have an
application where I generate around 1000 rows/s to be input into HBase. Then
I have to read this data and process it at regular intervals. Write speed is
not a problem as the cluster seems to be able to write at the reqd. rate.
But while processing this data also, I would need a read speed of at least
1000 rows/s since I need to keep the processing speed at least equal to the
data generation speed. So far, I am getting around 200-300 rows/s only it
seems. I have LZO compression on the tables and I haven't tried in-memory
yet as my RAM usage is too high already while running jobs. Is it possible
to achieve this read speed, and what can I do to improve it? How far can
adding more nodes/more RAM help? Please let me know if the scope is too huge
to answer this question and if you need more details.

Thanks,
Hari

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message