accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ACCUMULO-2036) Mappers are not running locally
Date Tue, 17 Dec 2013 00:02:08 GMT
Keith Turner created ACCUMULO-2036:
--------------------------------------

             Summary: Mappers are not running locally
                 Key: ACCUMULO-2036
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2036
             Project: Accumulo
          Issue Type: Bug
            Reporter: Keith Turner
             Fix For: 1.6.0


I ran listscans in the Accumulo shell while running continuous verify on small cluster and
almost not map task were running locally.

I think ACCUMULO-1585 has broken mapper locality in 1.6.0-SNAPSHOT.  Before that change Accumulo
would always store IP addrs.  Code like the following in o.a.a.c.client.mapreduce.AbstractInputFormat.getSplits()
would translate IPs to hostnames.  

{code:java}
       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getHostName();
          hostNameCache.put(ip, location);
        }
{code}

In my case I configured Accumulo to use hostnames, but not fully qualified ones.  So I think
the above code was just passing the non-quallified hostname through.   I suspected that maybe
hadoop wanted FQDN and changed the code the following and mappers ran locally.  I need to
confirm what hadoop is expecting.  I think the above code will result in a FQDN if given an
IP, but not a non-quallified hostname.  So this is not an issue for 1.4 or 1.5.

{code:java}
       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getCanonicalHostName();
          hostNameCache.put(ip, location);
        }
{code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message