hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
Date Tue, 10 Jan 2017 19:30:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815934#comment-15815934

Sean Busbey commented on HBASE-17437:

My read of the patch had the wal root dir default to the same as the hbase root door, and
that's fine by me; I have no desire to default the wal dir being outside of the hbase root
dir. but if it's set I'd prefer we save future heart ache by ensuring we aren't setting ourselves
up for writing within the hbase root dir in a way different from when this config isn't used.
Otherwise we run the risk of incorrect assumptions about what is written into various directories
within the hbase root dir. I would not want to rely on things being correct, for example,
if the wal root dir was set to the archive or oldWALs location. I don't know what future assumptions
we'll make about the hbase root dir, so better to just say "if you want to set this somewhere
else, it has to be somewhere entirely different than our root dir."

I don't have strong feelings on config naming; anything that accurately reflects the use is
fine. For example, {{hbase.wal.dir}} and {{hbase.wal.dir.perms}} would be accurate. A note
that the config is used both for RegionServer WALs and Master Procedure WALs would help avoid
future questions about why the permission isn't named as a regionserver-specific property.
Also, we should specify WALs are what's stored rather than logs, since "logs" has a multiple
meanings and in the context of HBase deploys usually refers to a human-consumable record of
service activity, e.g. as would be written to {{/var/log/}}.

> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>                 Key: HBASE-17437
>                 URL: https://issues.apache.org/jira/browse/HBASE-17437
>             Project: HBase
>          Issue Type: Improvement
>          Components: Filesystem Integration, wal
>    Affects Versions: 1.2.4
>            Reporter: Yishan Yang
>            Assignee: Yishan Yang
>              Labels: patch
>             Fix For: 2.0.0, 1.4.0
>         Attachments: hbase-17437-branch-1.2.patch, hbase-17437-master.patch
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some FileSystems
(such as Amazon S3) don’t support append or consistent writes. These two properties are
imperative for the WAL in order to avoid loss of writes. However, StoreFiles don’t necessarily
need the same consistency guarantees (since writes are cached locally and if writes fail,
they can always be replayed from the WAL).
> This JIRA aims to allow users to configure a log directory (for WALs) that is outside
of the root directory or even in a different FileSystem. The default value will still put
the log directory under the root directory.

This message was sent by Atlassian JIRA

View raw message