hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16578) Mob data loss after mob compaction and normal compaction
Date Wed, 19 Oct 2016 11:53:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15588540#comment-15588540
] 

Hudson commented on HBASE-16578:
--------------------------------

SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1814 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1814/])
HBASE-16578 Mob data loss after mob compaction and normal compaction (jingcheng.du: rev 67f1ac1f8e6c33e7917231ef2d541fe7461b4be4)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java


> Mob data loss after mob compaction and normal compaction
> --------------------------------------------------------
>
>                 Key: HBASE-16578
>                 URL: https://issues.apache.org/jira/browse/HBASE-16578
>             Project: HBase
>          Issue Type: Bug
>          Components: mob
>    Affects Versions: 2.0.0
>            Reporter: huaxiang sun
>            Assignee: Jingcheng Du
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16578-V2.patch, HBASE-16578-V3.patch, HBASE-16578.patch, TestMobCompaction.java,
TestMobCompaction.java
>
>
> StoreFileScanners on MOB cells rely on the scannerOrder to find the latest cells after
mob compaction. The value of scannerOrder is assigned by the order of maxSeqId of StoreFile,
and this maxSeqId is valued only after the reader of the StoreFile is created.
> In {{Compactor.compact}}, the compacted store files are cloned and their readers are
not created. And in {{StoreFileScanner.getScannersForStoreFiles}} the StoreFiles are sorted
before the readers are created and at that time the maxSeqId for each file is -1 (the default
value). This will lead  to a chaos in scanners in the following normal compaction. Some older
cells might be chosen during the normal compaction.
> We need to create readers either before the sorting in the method {{StoreFileScanner.getScannersForStoreFiles}},
or create readers just after the store files are cloned in {{Compactor.compact}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message