hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Shrivastava <rhshr...@gmail.com>
Subject Re: Proxy for HDFS
Date Tue, 10 Feb 2015 20:22:00 GMT
Thanks Rajiv. I will do the needful and loop back if I have more questions.

On Tue, Feb 10, 2015 at 8:01 AM, Rajiv Chittajallu <rajive@yahoo-inc.com>
wrote:

> And as for protecting certain fields, you may want to ask this HCatalog
> list.
>
> At Feb 9, 2015, 23:58:50, Rajiv Chittajallu<'rajive@yahoo-inc.com.INVALID'>
> wrote:
> read the documentation.
>
> ________________________________
>
> From: Rahul Shrivastava <rhshriva@gmail.com>
> To: Rajiv Chittajallu <rajive@yahoo-inc.com>
> Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
> Sent: Monday, February 9, 2015 9:53 PM
> Subject: Re: Proxy for HDFS
>
>
> Hi Rajiv,
>
> At runtime, can the user's program provides its credentials( username,
> password etc) and hdfs can verify it with the ACL entries ( username ,
> ofcourse password cannot be stored) to grant/deny it access to the file?
> So, the basic question is, can the user/group be provided at runtime rather
> using process owner username of the client process?
>
> Also, another reason for asking for Proxy is desire to control the access
> to the content ( example hide SSN, bank account etc) of the file rather
> than file itself. A proxy sitting in between client and datanode would
> achieve this as we could apply filter to the content at the proxy level.
>
> Please guide me as to if this feasible in current Hadoop architecture.
> Will an enhancement request to build a proxy hooks for HDFS so that we can
> apply more policy decisions at the proxy level make sense?
>
> thanks
> Rahul
>
>
> On Mon, Feb 9, 2015 at 4:53 PM, Rajiv Chittajallu <rajive@yahoo-inc.com>
> wrote:
>
> SOCKS proxies TCP connections. It wouldn't understand L7 traffic. New HDFS
> ACL feature[1] would provide additional controls. If there is a need beyond
> that, it would be better as a enhancement request in HDFS than building a
> proxy.
> >
> >
> >[1]
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_Access_Control_Lists
> >________________________________
> >From: Rahul Shrivastava <rhshriva@gmail.com>
> >To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu <rajive@yahoo-inc.com>
> >Sent: Monday, February 9, 2015 3:24 PM
> >Subject: Re: Proxy for HDFS
> >
> >Hi Rajiv,
> >
> >Thanks again for quick reply.
> >
> >The use case is as follows. Please let me know how can i use HDFS
> RPC+SOCKS proxy to achieve below.
> >
> >1. Client connect to HDFS proxy using HDFS driver ( not HTTPFs or
> WebHDFS).Proxy that sits between client and HDFS server.
> >2. Client request to access a file from HDFS cluster.
> >3. Proxy recognizes that the client does not have permission to access
> the file
> >4. Proxy replies back to the client with no data or access denied.
> >
> >Please note that policy that drives access control of the files is much
> more complex then just ACL of the file.
> >
> >thanks
> >rahul
> >
> >
> >
> >On Mon, Feb 9, 2015 at 3:11 PM, Rajiv Chittajallu <rajive@yahoo-inc.com>
> wrote:
> >
> >Rahul,
> >>
> >>If a client can use HDFS RPC why is a Proxy required? Are the clients
> not allowed to reach data nodes directly?
> >>
> >>WebHDFS + Apache Traffic server or HDFS RPC + SOCK Proxy should work.
> >>
> >>If you can share a use case, it would probably help.
> >>
> >>-rajive
> >>
> >>
> >>----- Original Message -----
> >>From: Rahul Shrivastava <rhshriva@gmail.com>
> >>To: hdfs-dev@hadoop.apache.org; Rajiv Chittajallu <rajive@yahoo-inc.com>
> >>Cc:
> >>
> >>Sent: Monday, February 9, 2015 2:58 PM
> >>Subject: Re: Proxy for HDFS
> >>
> >>Thanks Rajiv.
> >>
> >>I did look into HttpFs before but i wanted a build a proxy at HDFS layer.
> >>This is specific for clients which do not use HTTP ( i.e. HttpFs or
> >>webHDFS) to talk to the HDFS cluster. That includes clients which uses
> HDFS
> >>driver to talk to HDFS cluster.
> >>
> >>thanks
> >>Rahul
> >>
> >>
> >>
> >>
> >>
> >>On Mon, Feb 9, 2015 at 2:53 PM, Rajiv Chittajallu <
> >>rajive@yahoo-inc.com.invalid> wrote:
> >>
> >>>
> >>>
> https://github.com/apache/hadoop/tree/branch-2.6/hadoop-hdfs-project/hadoop-hdfs-httpfs
> >>>
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>> From: Rahul Shrivastava <rhshriva@gmail.com>
> >>> To: hdfs-dev@hadoop.apache.org
> >>> Cc:
> >>> Sent: Monday, February 9, 2015 2:35 PM
> >>> Subject: Proxy for HDFS
> >>>
> >>> Hi,
> >>>
> >>> I have looking in HDFS code base ( 2.6.0) for the last couple of days
> for
> >>> any possible proxy that we could utilize to create a proxy for HDFS.
> >>> Is there any hook within HDFS to build a proxy around that.
> >>>
> >>> thanks
> >>> Rahul
> >>>
> >>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message