Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (nike.apache.org: domain of josh.elser@gmail.com designates
 209.85.192.177 as permitted sender)
Message-ID: <53A30750.7070603@gmail.com>
Date: Thu, 19 Jun 2014 08:52:48 -0700
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: Is Data Locality Helpful? (or why run tserver and datanode on
 the same box?)
References: 
 <CAOiJXP4knYx8E-48KAbp__DsAhzPrQB85ikz5xe+Ycdm_7gzqQ@mail.gmail.com>
 <CAOHP_tFUr1_1WRBTOVxYy8MsgEB=iqffDrkA+9mXmSzidZTH9Q@mail.gmail.com>
In-Reply-To: 
 <CAOHP_tFUr1_1WRBTOVxYy8MsgEB=iqffDrkA+9mXmSzidZTH9Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

I believe this happens via the DfsClient, but you can only expect the 
first block of a file to actually be on the local datanode (assuming 
there is one). Everything else is possible to be remote. Assuming you 
have a proper rack script set up, you would imagine that you'll still 
get at least one rack-local replica (so you'd have a block nearby).

Interestingly (at least to me), I believe HBase does a bit of work in 
region (tablet) assignments to try to maximize the locality of regions 
WRT the datanode that is hosting the blocks that make up that file. I 
need to dig into their code some day though.

In general, Accumulo and HBase tend to be relatively comparable to one 
another with performance when properly configured which makes me apt to 
think that data locality can help, but it's not some holy grail (of 
course you won't ever hear me claim anything be in that position). I 
will say that I haven't done any real quantitative analysis either though.

tl;dr HDFS block locality should not be affecting the functionality of 
Accumulo.

On 6/19/14, 7:25 AM, Corey Nolet wrote:
> AFAIK, the locality may not be guaranteed right away unless the data for a
> tablet was first ingested on the tablet server that is responsible for that
> tablet, otherwise you'll need to wait for a major compaction to rewrite the
> RFiles locally on the tablet server. I would assume if the tablet server is
> not on the same node as the datanode, those files will probably be spread
> across the cluster as if you were ingesting data from outside the cloud.
>
> A recent discussion with Bill Slacum also brought to light a possible
> problem of the HDFS balancer [1] re-balancing blocks after the fact which
> could eventually pull blocks onto datanodes that are not local to the
> tablets. I believe remedy for this was to turn off the balancer or not have
> it run.
>
> [1]
> http://www.swiss-scalability.com/2013/08/hadoop-hdfs-balancer-explained.html
>
>
>
>
> On Thu, Jun 19, 2014 at 10:07 AM, David Medinets <david.medinets@gmail.com>
> wrote:
>
>> At the Accumulo Summit and on a recent client site, there have been
>> conversations about Data Locality and Accumulo.
>>
>> I ran an experiment to see that Accumulo can scan tables when the
>> tserver process is run on a server without a datanode process. I
>> followed these steps:
>>
>> 1. Start three node cluster
>> 2. Load data
>> 3. Kill datanode on slave1
>> 4. Wait until Hadoop notices dead node.
>> 5. Kill tserver on slave2
>> 6. Wait until Accumulo notices dead node.
>> 7. Run the accumulo shell on master and slave1 to verify entries can be
>> scanned.
>>
>> Accumulo handled this situation just fine. As I expected.
>>
>> How important (or not) is it to run tserver and datanode on the same
>> server?
>> Does the Data Locality implied by running them together exist?
>> Can the benefit be quantified?
>>
>