hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1476) Hive's metastore when run as a thrift service creates directories as the service user instead of the real user issuing create table/alter table etc.
Date Tue, 24 Aug 2010 18:02:17 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sushanth Sowmyan updated HIVE-1476:

    Attachment: HIVE-1476.patch

The above file attached for review is a patch generated by svn diff against revision 985768
off hive svn trunk.

It uses a new parameter "hadoop.fs.operations.owner" to determine whether or not a component
is allowed to perform operations(write) on the filesystem. It defaults to true for HiveMetaStore,
and defaults to false for HiveMetaStoreClient.

To run in thrift mode and get the behaviour described in this jira, we would need to override
this parameter to false on the thrift server start, and true on the client side.

Thanks for reviewing and looking over this!

Other comments : During testing, I wanted to disable filesystem access on the thrift server
end, by providing an invalid fs.default.name - the idea being that if this patch covered all
writing usecases, we would not need to write from the thrift end, and would not encounter
any runtime failures. However, as you may guess, that didn't turn out to be the case, because
we still read off it. We still perform path translations, we perform setLocation() on StorageDescriptors
after getting appropriate paths, etc. Now this raised another further issue, wherein we'd
potentially like this patch to be a bit more - to be able to move all fs ops to the client
side if configured to be so, as that would be useful for integrating with secure hadoop, where
the thrift server might not have access to the filesystem, and if it were to need access,
it would need to keep requesting auth tokens. The idea there might be to have it be a pure
metadata service. Anyhow, the intent now is to submit that as a separate feature-request jira,
while leaving this one to address this.

> Hive's metastore when run as a thrift service creates directories as the service user
instead of the real user issuing create table/alter table etc.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>                 Key: HIVE-1476
>                 URL: https://issues.apache.org/jira/browse/HIVE-1476
>             Project: Hadoop Hive
>          Issue Type: Bug
>    Affects Versions: 0.6.0, 0.7.0
>            Reporter: Pradeep Kamath
>         Attachments: HIVE-1476.patch
> If the thrift metastore service is running as the user "hive" then all table directories
as a result of create table are created as that user rather than the user who actually issued
the create table command. This is different semantically from non-thrift mode (i.e. local
mode) when clients directly connect to the metastore. In the latter case, directories are
created as the real user. The thrift mode should do the same.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message