hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laxman Ch <laxman....@gmail.com>
Subject Re: ReadOnly WebHDFS
Date Sat, 07 Nov 2015 05:18:52 GMT
Thanks for your response Chris.

>> I'm curious how this is a new problem after migration from HttpFs to
WebHDFS
With HttpFs, gateway admin users are allowed to proxy some limited set of
users.
They are not allowed to mock a root/dfsadmin user and do some fatal stuff
("rmr /").
With WebHDFS, thats not the case. I can send a request as any user
including root.

>> I would think this kind of thing could be achieved by writing a custom authentication
filter
Our changes to achieve this is also based on filter but that will get
activated based on configuration.
Just for reference, I added our filter code.

>> This is not a requirement I've heard from anyone else.
>> I'm generally reluctant to add features without a widespread need.
Agree. I wanted to listen to community before proceeding with jira/patch.
I thought this may be helpful for people running WebHDFS without security.
In worst case, I just need to maintain a inhouse patch till we enable
kerberos.

public class WebHdfsReadOnlyFilter implements ResourceFilter {

    public static final Log LOG =
LogFactory.getLog(WebHdfsReadOnlyFilter.class);
    public static final String DFS_WEBHDFS_READ_ONLY = "dfs.webhdfs.readonly";

    private boolean readonly;

    public WebHdfsReadOnlyFilter() {
        HdfsConfiguration conf = new HdfsConfiguration();
        readonly = conf.getBoolean(DFS_WEBHDFS_READ_ONLY, true);
        if(readonly) {
            LOG.info(DFS_WEBHDFS_READ_ONLY + " is set true. webhdfs is
readonly.");
        } else {
            LOG.warn(DFS_WEBHDFS_READ_ONLY + " is set false. write
operations are enabled over webhdfs");
        }
    }

    @Override
    public ContainerRequestFilter getRequestFilter() {
        return readonly ? READONLY_FILTER : null;
    }

    @Override
    public ContainerResponseFilter getResponseFilter() {
        return null;
    }

    private static final ContainerRequestFilter READONLY_FILTER = new
ContainerRequestFilter() {
        @Override
        public ContainerRequest filter(final ContainerRequest request) {
            if (!"GET".equals(request.getMethod())) {
                Response.ResponseBuilder builder = null;
                String response = "WebHDFS write operations are disabled.";
                // Though 405 (Method Not Allowed) looks more
appropriate, we could not use it due to unavailability in Jersey apis.
                // Please check apidocs for javax.ws.rs.core.Response.Status
                // Also HTTP spec says, we need to set allowed method in HEADER
                builder =
Response.status(Response.Status.FORBIDDEN).entity(response);
                throw new WebApplicationException(builder.build());
            }
            return request;
        }
    };

    @VisibleForTesting
    protected void setReadonly(boolean readonly) {
        this.readonly = readonly;
    }
}


On 6 November 2015 at 23:01, Chris Nauroth <cnauroth@hortonworks.com> wrote:

> Hello Laxman,
>
> I'm curious how this is a new problem after migration from HttpFs to
> WebHDFS.  With the HttpFs deployment architecture, were you somehow
> proxying only the read-only operations?
>
> I would think this kind of thing could be achieved by writing a custom
> authentication filter, deploying that to the HDFS classpath, and then
> pointing to it by setting dfs.web.authentication.filter in hdfs-site.xml
> to the full name of that custom authentication filter class.  The logic of
> the custom authentication filter would check for only read-only operations
> and reject the others.  This is a solution that wouldn't require changes
> in WebHDFS itself.
>
> This is not a requirement I've heard from anyone else.  I'm generally
> reluctant to add features without a widespread need.  Still, if you want
> to file an HDFS JIRA for further discussion of your proposal, there is no
> harm in that.  It might end up as a "Won't Fix", or perhaps others in the
> community see it differently from me, and we'd want to proceed.
>
> Thanks for sharing the work you've done!
>
> --Chris Nauroth
>
>
>
>
> On 11/6/15, 3:02 AM, "Laxman Ch" <laxman.lux@gmail.com> wrote:
>
> >Hi,
> >
> >We run a cluster with security set to simple.
> >Also, to some users, we had provided http access to HDFS via HttpFS
> >gateways.
> >However, this is not scaling and we are suffering from HttpFs gateway
> >choking problem. So, we wanted to enable WebHDFS directly on hadoop. But
> >this brings in the problem of security. Any user can simply delete
> >anything. And, we can't enable immediately enable kerberos security in
> >production.
> >
> >How about introducing a configuration to make WebHDFS readonly?
> >We patched this in our clusters cleanly and its working.
> >
> >Please revert with your comments if its a good idea to push this to
> >hadoop.
> >If yes, I will create a jira and submit patch.
> >--
> >Thanks,
> >Laxman
>
>


-- 
Thanks,
Laxman

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message