Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Tue, 17 Dec 2013 00:02:08 +0000 (UTC)
From: "Keith Turner (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12685065.1387238424860.46540.1387238528146@arcas>
In-Reply-To: <JIRA.12685065.1387238424860@arcas>
References: <JIRA.12685065.1387238424860@arcas>
Subject: [jira] [Created] (ACCUMULO-2036) Mappers are not running locally
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Keith Turner created ACCUMULO-2036:
--------------------------------------

             Summary: Mappers are not running locally
                 Key: ACCUMULO-2036
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2036
             Project: Accumulo
          Issue Type: Bug
            Reporter: Keith Turner
             Fix For: 1.6.0


I ran listscans in the Accumulo shell while running continuous verify on small cluster and almost not map task were running locally.

I think ACCUMULO-1585 has broken mapper locality in 1.6.0-SNAPSHOT.  Before that change Accumulo would always store IP addrs.  Code like the following in o.a.a.c.client.mapreduce.AbstractInputFormat.getSplits() would translate IPs to hostnames.  

{code:java}
       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getHostName();
          hostNameCache.put(ip, location);
        }
{code}

In my case I configured Accumulo to use hostnames, but not fully qualified ones.  So I think the above code was just passing the non-quallified hostname through.   I suspected that maybe hadoop wanted FQDN and changed the code the following and mappers ran locally.  I need to confirm what hadoop is expecting.  I think the above code will result in a FQDN if given an IP, but not a non-quallified hostname.  So this is not an issue for 1.4 or 1.5.

{code:java}
       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getCanonicalHostName();
          hostNameCache.put(ip, location);
        }
{code}


--
This message was sent by Atlassian JIRA
(v6.1.4#6159)