hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3147) The Namenode should be able to filter DN interfaces given to clients
Date Fri, 06 Apr 2012 00:08:29 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins updated HDFS-3147:
------------------------------

         Description: Not all DN interfaces exposed to clients from the NN should be used,
an interface may not be routable by the client, or a user may want to restrict off-cluster
clients from using cluster-private interfaces. Therefore the user should be able to make sure
clients may be given only a subset of the addresses reported by workers. This can be accomplished
by having masters filter the set of interfaces provided to clients, and/or having clients
filter the interfaces they're given. The former is preferable because the configuration resides
in a single place (the master instead of clients) and client configuration is less portable
(the configuration from an off-cluster client might end up getting used on-cluser if passed
as part of a job). In order to specify what interfaces clients receive the master is configured
with a table with rules that map a given source address range (of the incoming connection)
to a list of address ranges to used to filtering interfaces. An interface is given to the
client only if it matches one of the address ranges (for the given source address it came
in on). The rule has form:  Range -> list <Range>  where a range is specified in
CIDR notation. If a source address matches multiple entries in the table only the first rule
that matches is applied. If the table is empty or there are no matches then all interfaces
are given to the client.  (was: HDFS-3146 exposes multiple Datanode interfaces to the client.
However, not all interfaces exposed to clients should be used, eg because not all addresses
given to clients may be routable by the client, or a user may want to restrict off-cluster
clients from using cluster-private interfaces. Therefore the user should be able to configure
clients to use a subset of the addresses they are given. This can be accomplished by a new
configuration option ({{dfs.client.available.interfaces}}) that takes a list of interfaces
to use, interfaces that don't match the configuration are ignored. Acceptable configuration
values are the same as the {{dfs.datanode.available.interfaces}} parameter. In addition, we
could also add an option where clients automatically check if they can connect to each interface
that's given them, and filter those out by default.)
    Target Version/s: 2.0.0  (was: 0.23.3)
             Summary: The Namenode should be able to filter DN interfaces given to clients
 (was: The client should be able to specify which network interfaces to use)
    
> The Namenode should be able to filter DN interfaces given to clients
> --------------------------------------------------------------------
>
>                 Key: HDFS-3147
>                 URL: https://issues.apache.org/jira/browse/HDFS-3147
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>
> Not all DN interfaces exposed to clients from the NN should be used, an interface may
not be routable by the client, or a user may want to restrict off-cluster clients from using
cluster-private interfaces. Therefore the user should be able to make sure clients may be
given only a subset of the addresses reported by workers. This can be accomplished by having
masters filter the set of interfaces provided to clients, and/or having clients filter the
interfaces they're given. The former is preferable because the configuration resides in a
single place (the master instead of clients) and client configuration is less portable (the
configuration from an off-cluster client might end up getting used on-cluser if passed as
part of a job). In order to specify what interfaces clients receive the master is configured
with a table with rules that map a given source address range (of the incoming connection)
to a list of address ranges to used to filtering interfaces. An interface is given to the
client only if it matches one of the address ranges (for the given source address it came
in on). The rule has form:  Range -> list <Range>  where a range is specified in
CIDR notation. If a source address matches multiple entries in the table only the first rule
that matches is applied. If the table is empty or there are no matches then all interfaces
are given to the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message