hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18128) compaction marker could be skipped
Date Fri, 02 Jun 2017 12:41:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034624#comment-16034624

Anoop Sam John commented on HBASE-18128:

Ya, first we need a master branch patch.
On this patch as such there is no need to call WALEdit.getCompaction(kv)  to know whether
the Edit contains any Compaction marker cell. Just use WALEdit.isCompactionMarker(Cell).
Any other meta family cell will need similar treatment?  Can we make the solution in that
generic way?   May be a static method needed within WALEdit which takes the edit and check
for presence of any of such cells in it.   Is it possible that this kind of a meta cell getting
clubbed together with normal cell(s)?  I think no. Ted ?   I did not check code in detail.

> compaction marker could be skipped 
> -----------------------------------
>                 Key: HBASE-18128
>                 URL: https://issues.apache.org/jira/browse/HBASE-18128
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction, regionserver
>            Reporter: Jingyun Tian
>            Assignee: Jingyun Tian
>         Attachments: HBASE-18128.patch
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output files. Forces
sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The compaction
marker will be skipped when splitting log because the sequence id of compaction marker is
smaller than lastFlushedSequenceId.
> {code}
>         if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
>           editsSkipped++;
>           continue;
>         }
> {code}

This message was sent by Atlassian JIRA

View raw message