Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A668B9FA5 for ; Wed, 10 Oct 2012 20:04:49 +0000 (UTC) Received: (qmail 95986 invoked by uid 500); 10 Oct 2012 20:04:49 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 95951 invoked by uid 500); 10 Oct 2012 20:04:49 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 95941 invoked by uid 99); 10 Oct 2012 20:04:49 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 20:04:49 +0000 Received: from localhost (HELO mail-vc0-f169.google.com) (127.0.0.1) (smtp-auth username vines, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 20:04:48 +0000 Received: by mail-vc0-f169.google.com with SMTP id fl17so1501294vcb.0 for ; Wed, 10 Oct 2012 13:04:47 -0700 (PDT) Received: by 10.52.173.74 with SMTP id bi10mr11929396vdc.81.1349899487546; Wed, 10 Oct 2012 13:04:47 -0700 (PDT) MIME-Version: 1.0 Reply-To: vines@apache.org Received: by 10.220.63.5 with HTTP; Wed, 10 Oct 2012 13:04:27 -0700 (PDT) In-Reply-To: <0A8F66C42F49E448A40E99946404EE5B760B17723B@aplesstar.dom1.jhuapl.edu> References: <0A8F66C42F49E448A40E99946404EE5B760B17723B@aplesstar.dom1.jhuapl.edu> From: John Vines Date: Wed, 10 Oct 2012 16:04:27 -0400 Message-ID: Subject: Re: Writes and Loggers To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=bcaec51b1bb9700edf04cbb9f7ee --bcaec51b1bb9700edf04cbb9f7ee Content-Type: text/plain; charset=ISO-8859-1 Depending on the Instance your using, the client may hit HDFS first to get some instance information. Then either instance will hit Zookeeper to get the root tablet location. It will then scan that to find the appropriate !METADATA tablet, but will aggressively cache everything. The client will then scan (with heavy caching again) the !METADATA tablets necessary to find the tablet(s) for the range of writes. And then the writes will occur to appropriate tablets discovered in the !METADATA scan. Your order of operations for the write seems mostly correct. I think it's response from logger followed by insert into in memory map and then response to client. Eric knows this better than anyone though, so I'm going to defer in case I'm inaccurate. The scripts start loggers on all of the slave nodes. You can arbitrarily start/stop loggers, but we highly recommend doing 1:1 between them and tservers because the method for assigning tserver->logger does make assumptions depending on that 1:1 correlation. John On Wed, Oct 10, 2012 at 3:47 PM, Hider, Sandy wrote: > Hi all, > I want to verify some of the details with Accumulo Writes and Loggers. > > How does the client find the correct tablet server to write to? Does the > client query the master once and receive the ranges associated with each > tablet server? Is this list in zookeeper? > > With write-ahead loggers, when a client makes a write request, to a tablet > server: > The tablet server writes the data to a write-ahead logger which is > stored on a different slave > Then acknowledges the client > Then writes adds data to its in memory table. > > Do I have this order correct? > > Is the number of loggers always fixed, can this be changed with server > properties? > > Thanks in advance, > > Sandy > > > --bcaec51b1bb9700edf04cbb9f7ee Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Depending on the Instance your using, the client may hit HDFS first to get = some instance information. Then either instance will hit Zookeeper to get t= he root tablet location. It will then scan that to find the appropriate !ME= TADATA tablet, but will aggressively cache everything. The client will then= scan (with heavy caching again) the !METADATA tablets necessary to find th= e tablet(s) for the range of writes. And then the writes will occur to appr= opriate tablets discovered in the !METADATA scan.

Your order of operations for the write seems mostly correct. I think it= 's response from logger followed by insert into in memory map and then = response to client. Eric knows this better than anyone though, so I'm g= oing to defer in case I'm inaccurate.

The scripts start loggers on all of the slave nodes. You can arbitraril= y start/stop loggers, but we highly recommend doing 1:1 between them and ts= ervers because the method for assigning tserver->logger does make assump= tions depending on that 1:1 correlation.

John

On Wed, Oct 10, 2012 at 3:47 PM, Hider, Sandy <Sandy.= Hider@jhuapl.edu> wrote:
Hi all,
I want to verify some of the details with Accumulo Writes and Loggers.

How does the client find the correct tablet server to write to? =A0Does the= client query the master once and receive the ranges associated with each t= ablet server? Is this list in zookeeper?

With write-ahead loggers, when a client makes a write request, to a tablet = server:
=A0 =A0 =A0 =A0 The tablet server writes the data to a write-ahead logger w= hich is stored on a different slave
=A0 =A0 =A0 =A0 Then acknowledges the client
=A0 =A0 =A0 =A0 Then writes adds data to its in memory table.

Do I have this order correct?

Is the number of loggers always fixed, can this be changed with server prop= erties?

Thanks in advance,

Sandy



--bcaec51b1bb9700edf04cbb9f7ee--