hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fengnan Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14118) Use DNS to resolve Namenodes and Routers
Date Fri, 25 Jan 2019 07:46:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752002#comment-16752002

Fengnan Li commented on HDFS-14118:

[~csun] Thanks for the review
for 1, the current resolving logic is put into the helper that is called inside the getProxyAddress,
so if ObserverReadProxyProvider(or other proxy provider) wants to inherit this function, it
can take the resolving logic automatically. I did the current way to avoid code duplicate.

for 2, this is a good idea. I can drop the factory this way and loads the class from conf.

> Use DNS to resolve Namenodes and Routers
> ----------------------------------------
>                 Key: HDFS-14118
>                 URL: https://issues.apache.org/jira/browse/HDFS-14118
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Fengnan Li
>            Assignee: Fengnan Li
>            Priority: Major
>         Attachments: HDFS-14118.001.patch, HDFS-14118.patch
> Clients will need to know about routers to talk to the HDFS cluster (obviously), and
having routers updating (adding/removing) will have to make every client change, which is
a painful process.
> DNS can be used here to resolve the single domain name clients knows to a list of routers
in the current config. However, DNS won't be able to consider only resolving to the working
router based on certain health thresholds.
> There are some ways about how this can be solved. One way is to have a separate script
to regularly check the status of the router and update the DNS records if a router fails the
health thresholds. In this way, security might be carefully considered for this way. Another
way is to have the client do the normal connecting/failover after they get the list of routers,
which requires the change of current failover proxy provider.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message