hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
Date Thu, 02 Feb 2017 19:06:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850337#comment-15850337
] 

Andrew Purtell commented on HBASE-17437:
----------------------------------------

Belated +1 for the master commit.

Since the branch-1 patch is now up for review I looked at that. I see a couple of small nits
(e.g. naming of new conf variables) but they've already been committed to trunk. Bikeshedding
mostly, no matter. Some unit tests are not yet passing:
{noformat}
TestDefaultWALProviderWithHLogKey
TestWALReplay
TestDefaultWALProvider
TestWALReplayCompressed
TestSecureWALReplay
{noformat}

TestWALReplay.runWALSplit asserts WALSplitter.split should return a list of Paths containing
one entry, but the list is empty.

When the branch-1 patch is fixed so everything passes no reason not to get it in.

Thanks for contributing back [~zyork]

> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>
>                 Key: HBASE-17437
>                 URL: https://issues.apache.org/jira/browse/HBASE-17437
>             Project: HBase
>          Issue Type: Improvement
>          Components: Filesystem Integration, wal
>    Affects Versions: 1.2.4
>            Reporter: Yishan Yang
>            Assignee: Zach York
>              Labels: patch
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-17437.branch-1.001.patch, hbase-17437-branch-1.2.patch, HBASE-17437.master.001.patch,
HBASE-17437.master.002.patch, HBASE-17437.master.003.patch, HBASE-17437.master.004.patch,
HBASE-17437.master.005.patch, HBASE-17437.master.006.patch, HBASE-17437.master.007.patch,
HBASE-17437.master.008.patch, HBASE-17437.master.009.patch, HBASE-17437.master.010.patch,
HBASE-17437.master.011.patch, HBASE-17437.master.012.patch, hbase-17437-master.patch
>
>
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some FileSystems
(such as Amazon S3) don’t support append or consistent writes. These two properties are
imperative for the WAL in order to avoid loss of writes. However, StoreFiles don’t necessarily
need the same consistency guarantees (since writes are cached locally and if writes fail,
they can always be replayed from the WAL).
>  
> This JIRA aims to allow users to configure a log directory (for WALs) that is outside
of the root directory or even in a different FileSystem. The default value will still put
the log directory under the root directory.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message