Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Fri, 3 Jan 2014 05:46:55 +0000 (UTC)
From: "Josh Elser (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12685065.1387238424860.36430.1388728015702@arcas>
In-Reply-To: <JIRA.12685065.1387238424860@arcas>
References: <JIRA.12685065.1387238424860@arcas>
Subject: [jira] [Updated] (ACCUMULO-2036) Mappers are not running locally
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/ACCUMULO-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Elser updated ACCUMULO-2036:
---------------------------------

    Labels: 16_qa_bug  (was: )

> Mappers are not running locally
> -------------------------------
>
>                 Key: ACCUMULO-2036
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2036
>             Project: Accumulo
>          Issue Type: Bug
>            Reporter: Keith Turner
>            Assignee: Josh Elser
>              Labels: 16_qa_bug
>             Fix For: 1.6.0
>
>
> I ran listscans in the Accumulo shell while running continuous verify on small cluster and almost not map task were running locally.
> I think ACCUMULO-1585 has broken mapper locality in 1.6.0-SNAPSHOT.  Before that change Accumulo would always store IP addrs.  Code like the following in o.a.a.c.client.mapreduce.AbstractInputFormat.getSplits() would translate IPs to hostnames.  
> {code:java}
>        if (location == null) {
>           InetAddress inetAddress = InetAddress.getByName(ip);
>           location = inetAddress.getHostName();
>           hostNameCache.put(ip, location);
>         }
> {code}
> In my case I configured Accumulo to use hostnames, but not fully qualified ones.  So I think the above code was just passing the non-quallified hostname through.   I suspected hadoop wanted FQDN and changed the code to the following and mappers ran locally.  I need to confirm what hadoop is expecting.  I think the above code will result in a FQDN if given an IP,  so this is not an issue for 1.4 or 1.5.
> {code:java}
>        if (location == null) {
>           InetAddress inetAddress = InetAddress.getByName(ip);
>           location = inetAddress.getCanonicalHostName();
>           hostNameCache.put(ip, location);
>         }
> {code}


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)