hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5569) WebHDFS should support a deny/allow list for data access
Date Wed, 04 Dec 2013 17:54:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839127#comment-13839127

Alejandro Abdelnur commented on HDFS-5569:

bq. Why not use HttpFS/Hoop? .... Using a proxy comes with a lot of overhead and is not a
good solution for this request.

I guess you are not entirely familiar on how WebHDFS works, WebHDFS data transfers (read/write)
are done using the HDFS client API, so effectively the DN serving the whole file data transfer
is a PROXY and only saves network for blocks that happen to be in the same DN.

Because of this, the only difference between embedded WebHDFS and HttpFS is that WebHDFS is
not proxing metadata operations (NN only operations).

Also, based on performance tests done in the past, I did not see any performance degradation
because of the proxy.

bq. ... I'm unable to find references to HttpFS/Hoop in the 1.2.1 (stable) ....

Check HDFS-4262.

> WebHDFS should support a deny/allow list for data access
> --------------------------------------------------------
>                 Key: HDFS-5569
>                 URL: https://issues.apache.org/jira/browse/HDFS-5569
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>            Reporter: Adam Faris
>              Labels: features
> Currently we can't restrict what networks are allowed to transfer data using WebHDFS.
 Obviously we can use firewalls to block ports, but this can be complicated and problematic
to maintain.  Additionally, because all the jetty servlets run inside the same container,
blocking access to jetty to prevent WebHDFS transfers also blocks the other servlets running
inside that same jetty container.
> I am requesting a deny/allow feature be added to WebHDFS.  This is already done with
the Apache HTTPD server, and is what I'd like to see the deny/allow list modeled after.  

This message was sent by Atlassian JIRA

View raw message