hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
Date Fri, 16 Sep 2016 21:24:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15497422#comment-15497422
] 

Enis Soztutar commented on HBASE-16257:
---------------------------------------

Thanks Jerry for the latest patch. Sorry, but one last item may need to be addressed. Otherwise,
everything looks good. The HFileReplicator now uses the bulk load staging dir as a path under
root. This will become a problem if the source cluster is upgraded first before the sink cluster.
In case of cyclic replication, it will always be the case that you can have 2.x -> 1.x
replication. If that is the case, the staging dir will not be there under the 1.x root, which
will then case the HFileReplicator to fail. Maybe we can do a simple exists check on the remote
FS to see whether we can use the new hbase.root.dir/staging or we should fall back to using
the value from configuration. 

> Move staging dir to be under hbase root dir
> -------------------------------------------
>
>                 Key: HBASE-16257
>                 URL: https://issues.apache.org/jira/browse/HBASE-16257
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Jerry He
>            Assignee: Jerry He
>            Priority: Blocker
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch, HBASE-16257-v3.patch,
HBASE-16257-v4.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
>       + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir.  We are bringing secure bulkload
to the core. It makes sense to bring it under core control as well, instead of an optional
property.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message