hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17437) Support specifying a WAL directory outside of the root directory
Date Tue, 10 Jan 2017 16:02:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815354#comment-15815354
] 

Sean Busbey edited comment on HBASE-17437 at 1/10/17 4:02 PM:
--------------------------------------------------------------

can we ensure the wal root dir is not a sub-dir of the hbase root dir? (like, make sure folks
can't point it into {{hdfs:///hbase/tmp}}, {{hdfs:///hbase/data}}), {{hdfs:///hbase/archive}},
{{hdfs:///hbase/oldWALs}}, etc.)

Can we avoid adding the new constants into {{HConstants}}? Could they go e.g. into AbstractFSWALProvider
(which seems to incorrectly claim HDFS instead of Hadoop FileSystem)?

The config value for the WAL dir claims to be a regionserver config, but is also used by the
master for the procedure wal.


was (Author: busbey):
can we ensure the wal root dir is not a sub-dir of the hbase root dir? (like, make sure folks
can't point it into {{hdfs:///hbase/tmp}} or {{hdfs:///hbase/default}})

Can we avoid adding the new constants into {{HConstants}}? Could they go e.g. into AbstractFSWALProvider
(which seems to incorrectly claim HDFS instead of Hadoop FileSystem)?

The config value for the WAL dir claims to be a regionserver config, but is also used by the
master for the procedure wal.

> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>
>                 Key: HBASE-17437
>                 URL: https://issues.apache.org/jira/browse/HBASE-17437
>             Project: HBase
>          Issue Type: Improvement
>          Components: Filesystem Integration, wal
>    Affects Versions: 1.2.4
>            Reporter: Yishan Yang
>            Assignee: Yishan Yang
>              Labels: patch
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: hbase-17437-branch-1.2.patch, hbase-17437-master.patch
>
>
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some FileSystems
(such as Amazon S3) don’t support append or consistent writes. These two properties are
imperative for the WAL in order to avoid loss of writes. However, StoreFiles don’t necessarily
need the same consistency guarantees (since writes are cached locally and if writes fail,
they can always be replayed from the WAL).
>  
> This JIRA aims to allow users to configure a log directory (for WALs) that is outside
of the root directory or even in a different FileSystem. The default value will still put
the log directory under the root directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message