hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4039) Add option for WebHDFS to be a proxy, rather than just 302 to the DNs
Date Thu, 11 Oct 2012 10:03:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474004#comment-13474004
] 

Steve Loughran commented on HDFS-4039:
--------------------------------------

Based on my experience using webhdfs against remote EC2-hosted clusters, I can see the benefit
of allowing webhdfs to proxy to the DNs, rather than redirect to them. I know this could be
done with a separate proxy, but in a small5+ node cluster, this is one more thing to manage
-and in the small clusters the NN load is too low to matter.

In a remote cluster, the local user can access the Namenode via its external name if the cluster
security settings are set up to allow this. But WebHDFS 302s to the DNs via their internal
IPAddrs, not the external addresses, so the redirects don't work. There's also the issue that
you may not want to expose the internal DNs to the remote users, any more than you want the
IPC ports to be visible. 

If webhdfs could be turned into a proxy, its external IP address could be used for all get/post
operations, while the rest of the cluster was isolated from remote users. 
                
> Add option for WebHDFS to be a proxy, rather than just 302 to the DNs
> ---------------------------------------------------------------------
>
>                 Key: HDFS-4039
>                 URL: https://issues.apache.org/jira/browse/HDFS-4039
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 1.0.3
>            Reporter: Steve Loughran
>
> Add an option allowing webhdfs to proxy to the DNs, rather than redirecting to them.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message