hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17132) Cleanup deprecated code for WAL
Date Mon, 21 Nov 2016 19:56:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684561#comment-15684561
] 

Enis Soztutar commented on HBASE-17132:
---------------------------------------

Great, the patch is a almost identical to https://github.com/enis/hbase/commit/257d04d37c3ddbfd81ea0585217dbe4be8ab0b62
(HBASE-16475)  +  https://github.com/enis/hbase/commit/57e479d60c71a0e87e8b8b82f756bb8b4cf1e2c3
(HBASE-16476). One difference is Writer.getLength() or WAL.getFileLength() in my patch. But
this patch should be good to commit. I'll close those as duplicates after. 

In {{HbaseObjectWritableFor96Migration}} we still have to increment the {{code++}} otherwise
we won't be able to read following entries since they will be off by one written from older
code bases. 
See: 
https://github.com/enis/hbase/commit/57e479d60c71a0e87e8b8b82f756bb8b4cf1e2c3#diff-a20cbff2a97f4a66504bc6e316d66da9R232


WALSplitter.getReader() still throws CorruptedLogFileException. Why do we need this change?

{code}
-      try {
-        in = getReader(logfile, skipErrors, reporter);
-      } catch (CorruptedLogFileException e) {
-        LOG.warn("Could not get reader, corrupted log file " + logPath, e);
-        ZKSplitLog.markCorrupted(rootDir, logfile.getPath().getName(), fs);
-        isCorrupted = true;
-      }
+      in = getReader(logfile, skipErrors, reporter);
{code}

You can remove TestDefaultWALProviderWithHLogKey.java altogether, otherwise we will run the
same test twice. 

We were reading the PB WAL magic twice before? It seems to be the case where the WALFactory
opens the stream first, then the actual PBLogReader opens the stream again. If that is the
case, then this change is good: 
{code}
-            boolean isPbWal =
-                (stream.read(magic) == magic.length)
-                    && Arrays.equals(magic, ProtobufLogReader.PB_WAL_MAGIC);
-            reader = isPbWal ? new ProtobufLogReader() : new SequenceFileLogReader();
{code}


> Cleanup deprecated code for WAL
> -------------------------------
>
>                 Key: HBASE-17132
>                 URL: https://issues.apache.org/jira/browse/HBASE-17132
>             Project: HBase
>          Issue Type: Task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>              Labels: cleanup
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17132.patch
>
>
> There are some WAL related code which are marked as deprecated since branch-1(For example
the SequenceFileLogWriter). Let's remove it in 2.0 to keep the code clean.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message