hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingyun Tian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18128) compaction marker could be skipped
Date Wed, 07 Jun 2017 03:21:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040085#comment-16040085

Jingyun Tian commented on HBASE-18128:

[~anoop.hbase] [~tedyu]
Just submitted a patch and unit test for this issue. Please have a look.

I found there is a method OutputSink.keepRegionEvent(Entry entry) check if we should skip
the meta cell on the master branch, so I wrote the patch base on the method. Should I back
port it to earlier releases?

The unit test are almost same with TestWALSplit.testSplitLeavesCompactionEventsEdits(), the
only difference is the seqNum of compaction marker I set is -2. Since the default lastFlushedSequenceId
is -1, it will skip the compaction marker if we don't apply the patch.

> compaction marker could be skipped 
> -----------------------------------
>                 Key: HBASE-18128
>                 URL: https://issues.apache.org/jira/browse/HBASE-18128
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction, regionserver
>            Reporter: Jingyun Tian
>            Assignee: Jingyun Tian
>         Attachments: HBASE-18128-master.patch, TestCompactionMarker.java
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output files. Forces
sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The compaction
marker will be skipped when splitting log because the sequence id of compaction marker is
smaller than lastFlushedSequenceId.
> {code}
>         if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
>           editsSkipped++;
>           continue;
>         }
> {code}

This message was sent by Atlassian JIRA

View raw message