hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
Date Tue, 20 Sep 2016 01:18:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505232#comment-15505232
] 

Jerry He commented on HBASE-16257:
----------------------------------

bq.  If it is not the case, then we can commit this patch as it is.
Is this  +1  [~enis]?
 
Thanks for the comment, [~ashish singhi]

I tested with a local standalone instance and on a Keberos cluster. All looks good.
The directories are created or modified with the correct permissions:
{noformat}
hadoop fs -ls /apps/hbase/
drwx--x--x   - hbase hdfs          0 2016-09-19 13:48 /apps/hbase/data

hadoop fs -ls /apps/hbase/data
drwx------   - hbase hdfs          0 2016-09-19 11:25 /apps/hbase/data/.hbck
drwx------   - hbase hdfs          0 2016-09-19 13:48 /apps/hbase/data/.tmp
drwx------   - hbase hdfs          0 2016-09-19 18:01 /apps/hbase/data/MasterProcWALs
drwx------   - hbase hdfs          0 2016-09-19 13:48 /apps/hbase/data/WALs
drwx------   - hbase hdfs          0 2016-09-19 14:01 /apps/hbase/data/archive
drwx------   - hbase hdfs          0 2016-06-17 15:25 /apps/hbase/data/corrupt
drwx------   - hbase hdfs          0 2016-06-09 12:36 /apps/hbase/data/data
-rw-------   3 hbase hdfs         42 2016-06-09 12:35 /apps/hbase/data/hbase.id
-rw-------   3 hbase hdfs          7 2016-06-09 12:35 /apps/hbase/data/hbase.version
drwx------   - hbase hdfs          0 2016-09-19 11:25 /apps/hbase/data/mobdir
drwx------   - hbase hdfs          0 2016-09-19 17:59 /apps/hbase/data/oldWALs
drwx--x--x   - hbase hdfs          0 2016-09-19 17:34 /apps/hbase/data/staging
{noformat}
Ran a few bulk loads as well.
v5 adds back creating staging dir in SecureBulkLoadManager in the region server side if it
does not exist, so that it is ok to upgrade region server before master.

> Move staging dir to be under hbase root dir
> -------------------------------------------
>
>                 Key: HBASE-16257
>                 URL: https://issues.apache.org/jira/browse/HBASE-16257
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Jerry He
>            Assignee: Jerry He
>            Priority: Blocker
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch, HBASE-16257-v3.patch,
HBASE-16257-v4.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
>       + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir.  We are bringing secure bulkload
to the core. It makes sense to bring it under core control as well, instead of an optional
property.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message