hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11218) Data loss in HBase standalone mode
Date Thu, 29 May 2014 09:56:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012244#comment-14012244
] 

Hudson commented on HBASE-11218:
--------------------------------

SUCCESS: Integrated in HBase-0.98 #320 (See [https://builds.apache.org/job/HBase-0.98/320/])
HBASE-11218 Data loss in HBase standalone mode; REVERT (stack: rev 111ebaaea3bc70888ac27c09d0c8433f66df13e0)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java


> Data loss in HBase standalone mode
> ----------------------------------
>
>                 Key: HBASE-11218
>                 URL: https://issues.apache.org/jira/browse/HBASE-11218
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>             Fix For: 0.99.0
>
>         Attachments: 11218v2.098.txt, HBASE-11218-trunk-v1.diff, HBASE-11218-trunk-v2.diff
>
>
> Data loss in HBase standalone mode.
> *How to produce it*
> # Start HBase standalone mode.
> # Create a table using hbase shell.
> # Scan '.META.' and you will find data in meta table
> # Kill the HBase process with -9 option
> # Start the HBase agaion
> # Scan '.META.' and you will find nothing in meta table.
> *There are three main reasons.*
> # FSDataOutputStream.sync should call flush() if the underlying wrapped stream is not
Syncable. See HADOOP-8861
> # writeChecksum is ture in  default LocalFileSystem and the ChecksumFSOutputSummer will
buffer the data, which make the waledits are not written to os's filesystem with sync method
immediately, and those edits will be lost in regionserver's failover.
> #  The MiniZooKeeperCluster deletes the old zk data at startup which maye cause data
loss in meta table. The failover procedure is: split pre root regionserver's hlog -> assign
root -> split pre meta regionserver's hlog -> assign meta -> split all other regionservers'
hlogs -> assign other regions. If there is no data in zookeeper, we will get null for root
regionserver and then assign root table. Some data in root table maybe be lost for some root's
WalEdits have not been splited and replayed. So does the Meta table.
> I finished the patch for 0.94 and am working on the patch for trunk. Suggestions are
welcomed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message