hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
Date Mon, 12 Sep 2016 23:08:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15485581#comment-15485581
] 

Enis Soztutar commented on HBASE-16257:
---------------------------------------

Can we either use all upper case or all lower case for the names of these: 
{code}
+  private final FsPermission rootPerms = FsPermission.valueOf("-rwxr-xr-x");
+  // permissions for the files under rootDir that need secure protection
+  private final FsPermission secureRootFilePerms = FsPermission.valueOf("-rw-------");
+  // permissions for the directories under rootDir that need secure protection
+  private final FsPermission secureRootSubDirPerms;
+  // permissions for bulk load staging directory under rootDir
+  private final FsPermission PERM_HIDDEN = FsPermission.valueOf("-rwx--x--x");
{code}

Let's add a comment here talking about the layout of rootdir and the permissions of its subdirs.


This is now defined in two places: 
{code}
+  public static final String BULKLOAD_STAGING_DIR_NAME = "staging";
{code}

I was mentioning above that, for upgrades, we have to "open-up" the permissions for the root
directory because if a branch-1 cluster is run in secure mode, the root dir is already wiht
perms 700. After upgrade, the staging dir will be inside this root dir, and all BL operation
will fail after the upgrade. 
{code}
     // Filesystem is good. Go ahead and check for hbase.rootdir.
     try {
       if (!fs.exists(rd)) {
-        if (isSecurityEnabled) {
-          fs.mkdirs(rd, rootDirPerms);
-        } else {
-          fs.mkdirs(rd);
-        }
+        fs.mkdirs(rd);
{code}

Maybe extract the subdirs away, and add getters so that rest of the code does not have to
deal with parsing the conf again: 
{code}
+    checkSubDir(new Path(this.rootdir, HConstants.BASE_NAMESPACE_DIR));
+    checkSubDir(new Path(this.rootdir, HConstants.HFILE_ARCHIVE_DIRECTORY));
+    checkSubDir(new Path(this.rootdir, HConstants.HREGION_LOGDIR_NAME));
{code}

> Move staging dir to be under hbase root dir
> -------------------------------------------
>
>                 Key: HBASE-16257
>                 URL: https://issues.apache.org/jira/browse/HBASE-16257
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Jerry He
>            Assignee: Jerry He
>            Priority: Blocker
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch, HBASE-16257-v3.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
>       + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir.  We are bringing secure bulkload
to the core. It makes sense to bring it under core control as well, instead of an optional
property.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message