zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Esteban Gutierrez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-2249) CRC check failed when preAllocSize smaller than node data
Date Fri, 15 Dec 2017 16:45:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292784#comment-16292784

Esteban Gutierrez commented on ZOOKEEPER-2249:

There is something wrong on the FileTxnLog#append code path while padding the file, but we
have starting to see this more frequently on very large HBase cluster when HBase needs to
recovery very large replication queues and does massive Multi op (10s of MBs). 

> CRC check failed when preAllocSize smaller than node data
> ---------------------------------------------------------
>                 Key: ZOOKEEPER-2249
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2249
>             Project: ZooKeeper
>          Issue Type: Bug
>            Reporter: Benjamin Jaton
>              Labels: server
> Unexpected exception, exiting abnormally 
> java.io.IOException: CRC check failed 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:612)

> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157)

> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) 
> org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:272) 
> org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:399)
> To reproduce, set the preAllocSize to 8MB, the jute.maxbuffer to 20MB and try saving
a 15MB node several times.
> In my case the erroneous CRC appears after the second save. I use the LogFormatter class
to detect it.
> I suspect that the CRC error happens when the new transaction log is created, the code
probably expects to have enough room to save the transaction when creating a new file, but
it's too small.

This message was sent by Atlassian JIRA

View raw message