hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vihang Karajgaonkar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-16913) Support per-session S3 credentials
Date Fri, 16 Jun 2017 17:30:00 GMT
Vihang Karajgaonkar created HIVE-16913:
------------------------------------------

             Summary: Support per-session S3 credentials
                 Key: HIVE-16913
                 URL: https://issues.apache.org/jira/browse/HIVE-16913
             Project: Hive
          Issue Type: Improvement
            Reporter: Vihang Karajgaonkar
            Assignee: Vihang Karajgaonkar


Currently, the credentials needed to support Hive-on-S3 (or any other cloud-storage) need
to be to the hive-site.xml. Either using a hadoop credential provider or by adding the keys
in the hive-site.xml in plain text (unsecure)

This limits the usecase to using a single S3 key. If we configure per bucket s3 keys like
described [here | http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#Configurations_different_S3_buckets]
it exposes the access to all the buckets to all the hive users.

It is possible that there are different sets of users who would not like to share there buckets
and still be able to process the data using Hive. Enabling session level credentials will
help solve such use-cases. For example, currently this doesn't work

{noformat}
set fs.s3a.secret.key=my_secret_key;
set fs.s3a.access.key=my_access.key;
{noformat}

Because metastore is unaware of the the keys. This doesn't work either

{noformat}
set fs.s3a.secret.key=my_secret_key;
set fs.s3a.access.key=my_access.key;
set metaconf:fs.s3a.secret.key=my_secret_key;
set metaconf:fs.s3a.access.key=my_access_key;
{noformat}

This is because only a certain metastore configurations defined in {{HiveConf.MetaVars}} are
allowed to be set by the user. If we enable the above approaches we could potentially allow
multiple S3 credentials on a per-session level basis.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message