hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingyun Tian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-18128) compaction marker could be skipped
Date Tue, 13 Jun 2017 03:25:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jingyun Tian updated HBASE-18128:
---------------------------------
    Attachment: HBASE-18128-master-v3.patch
                TestCompactionMarker.java

[~tedyu] I didn't remove the condition, but to check if the cell is compaction marker. If
it is, I will set the lastFlushedSequenceId to Long.MAX_VALUE so I can avoid the compaction
marker being skipped.

{code}
        if (lastFlushedSequenceId >= seqId) {
           editsSkipped++;
           continue;
         }
{code}

Patch updated, please check it out.
Thanks

> compaction marker could be skipped 
> -----------------------------------
>
>                 Key: HBASE-18128
>                 URL: https://issues.apache.org/jira/browse/HBASE-18128
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction, regionserver
>            Reporter: Jingyun Tian
>            Assignee: Jingyun Tian
>         Attachments: HBASE-18128-master.patch, HBASE-18128-master-v2.patch, HBASE-18128-master-v3.patch,
TestCompactionMarker.java
>
>
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output files. Forces
sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The compaction
marker will be skipped when splitting log because the sequence id of compaction marker is
smaller than lastFlushedSequenceId.
> {code}
>         if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
>           editsSkipped++;
>           continue;
>         }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message