hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingyun Tian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18128) compaction marker could be skipped
Date Wed, 14 Jun 2017 11:02:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049053#comment-16049053
] 

Jingyun Tian commented on HBASE-18128:
--------------------------------------

The failed test is unrealted to my patch and it can pass on my computer.

[~allan163] I think the point is we add the compaction marker is to guarantee the old file
won't show up. If we don't need to guarantee this, maybe the compaction marker we can also
skip during splitting log.

> compaction marker could be skipped 
> -----------------------------------
>
>                 Key: HBASE-18128
>                 URL: https://issues.apache.org/jira/browse/HBASE-18128
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction, regionserver
>            Reporter: Jingyun Tian
>            Assignee: Jingyun Tian
>         Attachments: HBASE-18128-master.patch, HBASE-18128-master-v2.patch, HBASE-18128-master-v3.patch,
TestCompactionMarker.java
>
>
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output files. Forces
sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The compaction
marker will be skipped when splitting log because the sequence id of compaction marker is
smaller than lastFlushedSequenceId.
> {code}
>         if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
>           editsSkipped++;
>           continue;
>         }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message