hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8198) Support multiple network interfaces
Date Thu, 12 Apr 2012 14:43:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252465#comment-13252465

Daryn Sharp commented on HADOOP-8198:

@Todd: If the scope is only DNs, then you may be correct that it's not a blocking issue. 
The basic issue is a token cannot be acquired over 1 interface, and then subsequently used
via another.  This may or may not be an issue today, but it's worth noting that it places
strong limitations on clients and network topologies in a multi-NIC environment.

We have to ensure that all services issuing tokens to an external client cannot be cannot
be run on a multi-NIC host.  I believe this currently applies to the NN, RM, AM(??), JHS,
and maybe others.  Since the AM runs on a DN, it's the one to be most concerned about.  I
don't understand the yarn token passing enough to know if it is a problem today.

Repurposing of hosts will be impacted.  If a service is moved from a failed host to another
host, and the new host is multi-NIC, then the grid internal interfaces must be shutdown. 
If the host is repurposed to be a DN again, then the interfaces will need to be re-enabled.

> Support multiple network interfaces
> -----------------------------------
>                 Key: HADOOP-8198
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8198
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io, performance
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: MultipleNifsv1.pdf, MultipleNifsv2.pdf, MultipleNifsv3.pdf
> Hadoop does not currently utilize multiple network interfaces, which is a common user
request, and important in enterprise environments. This jira covers a proposal for enhancements
to Hadoop so it better utilizes multiple network interfaces. The primary motivation being
improved performance, performance isolation, resource utilization and fault tolerance. The
attached design doc covers the high-level use cases, requirements, a proposal for trunk/0.23,
discussion on related features, and a proposal for Hadoop 1.x that covers a subset of the
functionality of the trunk/0.23 proposal.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message