hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8198) Support multiple network interfaces
Date Wed, 11 Apr 2012 13:55:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251573#comment-13251573
] 

Nathan Roberts commented on HADOOP-8198:
----------------------------------------

Hi Eli, I'd really like to understand the additional complexity that this feature will add
to the overall system. Not only in the hadoop software itself, but also in the management
of the cluster. Some examples that come to mind: 
* More configuration to get right/wrong, not only at the hadoop level but also at the host
and network
* More physical wires to get crossed
* More components that can now fail - nodes can have 1 of 2 nics fail - what will that mean?
How much more software will have to be written to deal with cases like this?
* Considerably more cluster configurations that will need to be tested. 
* Token handling gets even trickier, even beyond what will be needed for HA, I think??

Given the fact that 10g ethernet is right around the corner (like very price competitive this
year), what does that do to the equation? 

Assuming host-level bonding was in place, what are the real benefits we would see on clusters
made up of commodity boxes (no separate high-speed fabric)?

Anything you can do to help clarify/quantify/address the additional complexity will help.




                
> Support multiple network interfaces
> -----------------------------------
>
>                 Key: HADOOP-8198
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8198
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io, performance
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: MultipleNifsv1.pdf, MultipleNifsv2.pdf, MultipleNifsv3.pdf
>
>
> Hadoop does not currently utilize multiple network interfaces, which is a common user
request, and important in enterprise environments. This jira covers a proposal for enhancements
to Hadoop so it better utilizes multiple network interfaces. The primary motivation being
improved performance, performance isolation, resource utilization and fault tolerance. The
attached design doc covers the high-level use cases, requirements, a proposal for trunk/0.23,
discussion on related features, and a proposal for Hadoop 1.x that covers a subset of the
functionality of the trunk/0.23 proposal.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message