accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: MR Data Locality with AccumuloInputFormat?
Date Fri, 16 May 2014 20:37:14 GMT
Hi Russ,

I believe that the AccumuloInputFormat will use the splits on the table 
you're reading to generate the MR InputSplits. The InputFormat should be 
trying to run the Mappers on the same machine as the tserver serving the 
data is located.

If you're only getting a few mappers, adding more splits to your table 
should help. As your job runs, you can verify locality using the 
counters that your Job creates using the JobTracker/ResourceManger web UI.

On 5/16/14, 1:32 PM, Russ Weeks wrote:
> Hi, folks,
>
> When I execute an MR job with AccumuloInputFormat, are there any
> guarantees about which mappers process which rows? I'm trying to
> minimize crosstalk in my cluster but either I haven't split my table
> properly or I'm expecting too much, because I'm only seeing 1 or 2 nodes
> running MR tasks that should be reading data from tablet servers on 8
> different nodes.
>
> Thanks,
> -Russ

Mime
View raw message