hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Lord (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-6422) Add switch in LoadIncrementalHFiles API to allow for programatically changing
Date Wed, 18 Jul 2012 19:13:34 GMT
Jeff Lord created HBASE-6422:
--------------------------------

             Summary: Add switch in LoadIncrementalHFiles API to allow for programatically
changing 
                 Key: HBASE-6422
                 URL: https://issues.apache.org/jira/browse/HBASE-6422
             Project: HBase
          Issue Type: New Feature
          Components: mapred
    Affects Versions: 0.94.0
            Reporter: Jeff Lord


Hbase bulk load often requires manual chown and permission changes.
Usually it goes something like try to run completebulkload and it fails with the following
hdfs error:

org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException:
Permission denied:  user=hbase, access=WRITE, inode="mydata":
hadoop:supergroup:rwxr-xr-x

To work around this mismatch between Hadoop and HBase user permissions, you can make both
users share a group: that is, the user where you run the MapReduce jobs and the user running
HBase. Then, after running your MapReduce job, you can chgrp the output directory to the HBase
group, and run chmod g+w. This allows the bulk loader to move the files into the HBase data
directory.

It would be useful if there was a way to do this in the LoadIncrementalHFiles API

It is the case of linux permissions too:
If we have:
file owner:me group:me
We can chown to group:x and owner:x, without needing special permissions
chown x:x file

Then when we trigger bulk load, HBase does the fs -mv, and finds that the owner is itself,
so no permission hitches. Goes smooth.

We are thinking that a switch to turn this on/off (default off) would be nice to have.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message