hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot West <tea...@gmail.com>
Subject Securing secrets for S3 FileSystems in DistCp
Date Tue, 03 May 2016 12:41:21 GMT

We're currently using DistCp and S3 FileSystems to move data from a vanilla
Apache Hadoop cluster to S3. We've been concerned about exposing our AWS
secrets on our shared, on-premise cluster. As  a work-around we've patched
DistCp to load these secrets from a JCEKS keystore. This seems to work
quite well, however we're not comfortable on relying on a DistCp fork.

What is the usual approach to achieve this with DistCp and is there a
feature or practice that we've overlooked? If not, might there be value in
us raising a JIRA ticket and submitting a patch for DistCp to include this
secure keystore functionality?

Thanks - Elliot.

View raw message