Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of ryanobjc@gmail.com designates
 209.85.210.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=Rmly1xOpryBHeQYnvwf8gqB5Wfj6K54Tk99Sb9ZU5S3nzHDKpWUKWa+1Sv3hzbgMya
         Qm2zDhAIVfo8ZC9je10sQYcFBTw2Vg/LlEk/DKYDp4Ts5N0P8fPa6LQgOACICKY61JnX
         o3PfDqg6IPdt+dX+nJ3q7mesUvFffaymloD38=
MIME-Version: 1.0
In-Reply-To: 
 <7CA0D5FE7FA83048893E9230C1E9C0280C444921@DNVREMSA01.jsq.bsg.ad.adp.com>
References: 
 <7CA0D5FE7FA83048893E9230C1E9C0280C3DC637@DNVREMSA01.jsq.bsg.ad.adp.com>
	<AANLkTik=NuT0HZ7F=X3SkxG3ipsEGAB+m0gjRL=L6rC-@mail.gmail.com>
	<7CA0D5FE7FA83048893E9230C1E9C0280C444921@DNVREMSA01.jsq.bsg.ad.adp.com>
Date: Fri, 25 Feb 2011 14:11:56 -0800
Message-ID: <AANLkTin=Gy_rnD26g-BXpc2HgwuL+zPMkN2dS5Dxk86j@mail.gmail.com>
Subject: Re: when does put return to the caller?
From: Ryan Rawson <ryanobjc@gmail.com>
To: user@hbase.apache.org
Cc: "Hiller, Dean (Contractor)" <dean.hiller@broadridge.com>,
 tsuna <tsunanet@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I think we have a category error here, perhaps you should read the
bigtable paper which succinctly describes the overall architecture.
It is here: http://labs.google.com/papers/bigtable.html

Your statements dont really make sense in the context of the
bigtable/hbase architecture.  When you write, the write goes to a
single server which then writes to WAL.  There is the
deferred_log_flush option for tables to cause edits for said table to
be done in the background as well.


On Fri, Feb 25, 2011 at 1:58 PM, Hiller, Dean  (Contractor)
<dean.hiller@broadridge.com> wrote:
> Is there any work being done in that I could have my puts write to 3 node=
s in-memory(which asynchronously write to the WAL)? =A0Is that a possibilit=
y? =A0Then if 1 or 2 nodes go down, I can still recover. =A0Obviously if al=
l 3 go down, I would have a problem.
>
> Thanks,
> Dean
>
> -----Original Message-----
> From: tsuna [mailto:tsunanet@gmail.com]
> Sent: Thursday, February 24, 2011 10:05 AM
> To: user@hbase.apache.org
> Cc: Hiller, Dean (Contractor)
> Subject: Re: when does put return to the caller?
>
> On Wed, Feb 23, 2011 at 1:25 PM, Hiller, Dean =A0(Contractor)
> <dean.hiller@broadridge.com> wrote:
>> I was wonder if put returns after writing the data into memory on two
>> out of the three nodes letting my client continue so we don't have to
>> wait for the memory to then go to disk. =A0After all, if it is replicate=
d,
>> probably don't need to wait for it to be written to disk(ie. Kind of
>> like the in-memory data grids that exist out there).
>
> If you use the WAL (Write Ahead Log), which is enabled by default,
> your write has to be persisted on 3 disks before it returns
> successfully to you. =A0Without the WAL, the write is only written to
> memory of one node, so if that node crashes, you'll lose your edit
> (much faster but unsafe).
>
>> Also, is there an asynchronous request/response for PutAll so I can slam
>> the grid with batches of 1000 entries kind of like
>>
>> PutAll(List<PutOps> puts, AsynchCallback cb); such that cb would be
>> called with the failure or success response after the put?
>
> HBase doesn't offer any asynchronous API out of the box, and in
> addition HTable isn't thread-safe. =A0If you want an asynchronous HBase
> API, I recommend you take a look at asynchbase
> (https://github.com/stumbleupon/asynchbase) as it's an alternative
> HBase client that is entirely asynchronous and non-blocking. =A0Javadoc
> is at http://su.pr/1PJCSY
>
> --
> Benoit "tsuna" Sigoure
> Software Engineer @ www.StumbleUpon.com
> This message and any attachments are intended only for the use of the add=
ressee and
> may contain information that is privileged and confidential. If the reade=
r of the
> message is not the intended recipient or an authorized representative of =
the
> intended recipient, you are hereby notified that any dissemination of thi=
s
> communication is strictly prohibited. If you have received this communica=
tion in
> error, please notify us immediately by e-mail and delete the message and =
any
> attachments from your system.
>
>