hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot West <tea...@gmail.com>
Subject Re: Securing secrets for S3 FileSystems in DistCp
Date Tue, 03 May 2016 12:55:05 GMT
Thanks for your reply.

We have IAM users, each with their own sets of keys. Could you explain how
I can use roles in this situation?


On 3 May 2016 at 13:46, Shekhar Sharma <shekhar2581@gmail.com> wrote:

> Have u used  IAM (identity  access management ) roles ?
> On 3 May 2016 18:11, "Elliot West" <teabot@gmail.com> wrote:
>> Hello,
>> We're currently using DistCp and S3 FileSystems to move data from a
>> vanilla Apache Hadoop cluster to S3. We've been concerned about exposing
>> our AWS secrets on our shared, on-premise cluster. As  a work-around we've
>> patched DistCp to load these secrets from a JCEKS keystore. This seems to
>> work quite well, however we're not comfortable on relying on a DistCp fork.
>> What is the usual approach to achieve this with DistCp and is there a
>> feature or practice that we've overlooked? If not, might there be value in
>> us raising a JIRA ticket and submitting a patch for DistCp to include this
>> secure keystore functionality?
>> Thanks - Elliot.

View raw message