hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean (Contractor)" <dean.hil...@broadridge.com>
Subject asynch api mailing list?
Date Sun, 27 Feb 2011 18:29:23 GMT
I had a question on the hbase asynch api.

 

I find it odd that there is a PleaseThrottleException as on most asynch
systems I have worked on, each node has a queue and when the queue fills
up, the nic buffer in it fills up which means the remote nic buffer then
fills up and then the client should be blocking all writes which means
his incoming buffer fills up, etc. etc. (or he can spin needlessly but
that is usually a very very bad choice...in fact, I have never seen that
work out well and always was reverted).

 

The nice thing I always found with asynch systems is you completely
control your memory footprint of the server with the incoming
queue...direct relationship between that queue size and the memory of
the node used.  The other things that was always done was asynch reads
but always do synch writes(or block on a lock if write can't go through
until it can go through to slow down the upstream system and throttle
it).....ie. it is a self throttling system when done this way so there
is no need for a PleaseThrottleException which I find odd.

 

Maybe I am missing something though????  As a client, I definitely want
async reads but my writes should only return if there was room in the
nic buffer to write it out, otherwise it should block and hold up my
client so my client doesn't have to do any extra coding for a
PleaseThrottleException.

 

In this solution, most of my writes will be asynch right up until hbase
starts becoming a bottleneck.

 

Thanks,

Dean

 

 


This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message