hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Swarnim Kulkarni (JIRA)" <>
Subject [jira] [Commented] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories
Date Thu, 29 May 2014 04:24:02 GMT


Swarnim Kulkarni commented on HIVE-7116:

Hey Vikram,

Looking deeper into the patch, in my opinion there could be some issues:

1. "hadoop-1" pulls in the 1.2.1 hadoop jar. The FileSystem object on this doesn't have the
newInstance method on it. So with "-P hadoop-1" this would fail with a compilation error.
2. I think the latest patch is overly complicated for what it is trying to do. I think it
could be simplified to just the following:

FileSystem fs = FileSystem.newInstance(mkdirPath.toUri(), conf);
    boolean retval = false;
    try {
      retval = fs.mkdirs(mkdirPath, fsPermission);
    } finally {;

3. Are we sure that it is the cached FileSystem instance that is causing the permissions to
not get reflected? I wrote a simple test with cached FileSystem object[1] that sets permissions
and it passed for me. Interestingly it only fails for "777" permissions so might be some issue
specific to that additional "executable" sticky bit.

Just my 0.02 :)


> HDFS FileSystem object cache causes permission issues in creating tmp directories
> ---------------------------------------------------------------------------------
>                 Key: HIVE-7116
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2, Tez
>    Affects Versions: 0.13.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>         Attachments: HIVE-7116.1.patch, HIVE-7116.2.patch, HIVE-7116.3.patch, HIVE-7116.4.patch
> We change permissions of the directory creation to 777 for HiveServer 2 operation and
it turns out that because of HDFS caching, it does not reflect once created. We need to use
the non-cached version of the API to get a FileSystem object to fix this.

This message was sent by Atlassian JIRA

View raw message