hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: when does put return to the caller?
Date Fri, 25 Feb 2011 22:11:56 GMT
I think we have a category error here, perhaps you should read the
bigtable paper which succinctly describes the overall architecture.
It is here: http://labs.google.com/papers/bigtable.html

Your statements dont really make sense in the context of the
bigtable/hbase architecture.  When you write, the write goes to a
single server which then writes to WAL.  There is the
deferred_log_flush option for tables to cause edits for said table to
be done in the background as well.



On Fri, Feb 25, 2011 at 1:58 PM, Hiller, Dean  (Contractor)
<dean.hiller@broadridge.com> wrote:
> Is there any work being done in that I could have my puts write to 3 nodes in-memory(which
asynchronously write to the WAL)?  Is that a possibility?  Then if 1 or 2 nodes go down,
I can still recover.  Obviously if all 3 go down, I would have a problem.
>
> Thanks,
> Dean
>
> -----Original Message-----
> From: tsuna [mailto:tsunanet@gmail.com]
> Sent: Thursday, February 24, 2011 10:05 AM
> To: user@hbase.apache.org
> Cc: Hiller, Dean (Contractor)
> Subject: Re: when does put return to the caller?
>
> On Wed, Feb 23, 2011 at 1:25 PM, Hiller, Dean  (Contractor)
> <dean.hiller@broadridge.com> wrote:
>> I was wonder if put returns after writing the data into memory on two
>> out of the three nodes letting my client continue so we don't have to
>> wait for the memory to then go to disk.  After all, if it is replicated,
>> probably don't need to wait for it to be written to disk(ie. Kind of
>> like the in-memory data grids that exist out there).
>
> If you use the WAL (Write Ahead Log), which is enabled by default,
> your write has to be persisted on 3 disks before it returns
> successfully to you.  Without the WAL, the write is only written to
> memory of one node, so if that node crashes, you'll lose your edit
> (much faster but unsafe).
>
>> Also, is there an asynchronous request/response for PutAll so I can slam
>> the grid with batches of 1000 entries kind of like
>>
>> PutAll(List<PutOps> puts, AsynchCallback cb); such that cb would be
>> called with the failure or success response after the put?
>
> HBase doesn't offer any asynchronous API out of the box, and in
> addition HTable isn't thread-safe.  If you want an asynchronous HBase
> API, I recommend you take a look at asynchbase
> (https://github.com/stumbleupon/asynchbase) as it's an alternative
> HBase client that is entirely asynchronous and non-blocking.  Javadoc
> is at http://su.pr/1PJCSY
>
> --
> Benoit "tsuna" Sigoure
> Software Engineer @ www.StumbleUpon.com
> This message and any attachments are intended only for the use of the addressee and
> may contain information that is privileged and confidential. If the reader of the
> message is not the intended recipient or an authorized representative of the
> intended recipient, you are hereby notified that any dissemination of this
> communication is strictly prohibited. If you have received this communication in
> error, please notify us immediately by e-mail and delete the message and any
> attachments from your system.
>
>

Mime
View raw message