hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zach York (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17437) Support specifying a WAL directory outside of the root directory
Date Fri, 27 Jan 2017 23:09:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843630#comment-15843630
] 

Zach York commented on HBASE-17437:
-----------------------------------

[~enis] [~stack] Thanks for the reviews. I will update the patch today. Also, thanks for explaining
the fs redo project. I would love to help with that effort (and try to get some of it out
by HBase 2.0 if I have some good reviewers ;) ). 

I have been thinking about moving everything to HFileSystem and have a couple approaches:
1) Naive approach: Move the getWALRootDirectory() and getWALFileSystem() to HFileSystem and
use them as they are being used right now.
This way is error-prone because people can forget to use the right FileSystem in their changes
(after this change) and break things.

2) Intelligent approach: Have HFileSystem override all the methods from FileSystem and ensure
that the correct FileSystem is used based off of the scheme.
This way is much better, but might incur a small performance loss (should be negligible though).

What do you guys think?
I will be implementing approach 1 to move complete this JIRA, but then would like to do approach
2 in a follow-up JIRA.

> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>
>                 Key: HBASE-17437
>                 URL: https://issues.apache.org/jira/browse/HBASE-17437
>             Project: HBase
>          Issue Type: Improvement
>          Components: Filesystem Integration, wal
>    Affects Versions: 1.2.4
>            Reporter: Yishan Yang
>            Assignee: Zach York
>              Labels: patch
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: hbase-17437-branch-1.2.patch, HBASE-17437.master.001.patch, HBASE-17437.master.002.patch,
HBASE-17437.master.003.patch, HBASE-17437.master.004.patch, hbase-17437-master.patch
>
>
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some FileSystems
(such as Amazon S3) don’t support append or consistent writes. These two properties are
imperative for the WAL in order to avoid loss of writes. However, StoreFiles don’t necessarily
need the same consistency guarantees (since writes are cached locally and if writes fail,
they can always be replayed from the WAL).
>  
> This JIRA aims to allow users to configure a log directory (for WALs) that is outside
of the root directory or even in a different FileSystem. The default value will still put
the log directory under the root directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message