Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Thu, 2 Jun 2016 14:33:59 +0000 (UTC)
From: "Martin Nowak (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12973550.1464380478000.15646.1464878039324@Atlassian.JIRA>
In-Reply-To: <JIRA.12973550.1464380478000@Atlassian.JIRA>
References: <JIRA.12973550.1464380478000@Atlassian.JIRA> <JIRA.12973550.1464380478339@arcas>
Subject: [jira] [Commented] (KAFKA-3764) Error processing append operation
 on partition
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 02 Jun 2016 14:34:01 -0000


    [ https://issues.apache.org/jira/browse/KAFKA-3764?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1531=
2402#comment-15312402 ]=20

Martin Nowak commented on KAFKA-3764:
-------------------------------------

Good guess, there is a bug report for snappy-java w/ a similar Kafka stackt=
race.
[bad handling of the MAGIC HEADER =C2=B7 Issue #142 =C2=B7 xerial/snappy-ja=
va](https://github.com/xerial/snappy-java/issues/142)

> Error processing append operation on partition
> ----------------------------------------------
>
>                 Key: KAFKA-3764
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3764
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.10.0.0
>            Reporter: Martin Nowak
>
> After updating Kafka from 0.9.0.1 to 0.10.0.0 I'm getting plenty of `Erro=
r processing append operation on partition` errors. This happens with ruby-=
kafka as producer and enabled snappy compression.
> {noformat}
> [2016-05-27 20:00:11,074] ERROR [Replica Manager on Broker 2]: Error proc=
essing append operation on partition m2m-0 (kafka.server.ReplicaManager)
> kafka.common.KafkaException:=20
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBuffer=
MessageSet.scala:159)
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBuffer=
MessageSet.scala:85)
>         at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate=
.scala:64)
>         at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:56=
)
>         at kafka.message.ByteBufferMessageSet$$anon$2.makeNextOuter(ByteB=
ufferMessageSet.scala:357)
>         at kafka.message.ByteBufferMessageSet$$anon$2.makeNext(ByteBuffer=
MessageSet.scala:369)
>         at kafka.message.ByteBufferMessageSet$$anon$2.makeNext(ByteBuffer=
MessageSet.scala:324)
>         at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate=
.scala:64)
>         at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:56=
)
>         at scala.collection.Iterator$class.foreach(Iterator.scala:893)
>         at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30=
)
>         at kafka.message.ByteBufferMessageSet.validateMessagesAndAssignOf=
fsets(ByteBufferMessageSet.scala:427)
>         at kafka.log.Log.liftedTree1$1(Log.scala:339)
>         at kafka.log.Log.append(Log.scala:338)
>         at kafka.cluster.Partition$$anonfun$11.apply(Partition.scala:443)
>         at kafka.cluster.Partition$$anonfun$11.apply(Partition.scala:429)
>         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
>         at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:237)
>         at kafka.cluster.Partition.appendMessagesToLeader(Partition.scala=
:429)
>         at kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(=
ReplicaManager.scala:406)
>         at kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(=
ReplicaManager.scala:392)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(Traversa=
bleLike.scala:234)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(Traversa=
bleLike.scala:234)
>         at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(Hash=
Map.scala:99)
>         at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(Hash=
Map.scala:99)
>         at scala.collection.mutable.HashTable$class.foreachEntry(HashTabl=
e.scala:230)
>         at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40=
)
>         at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
>         at scala.collection.TraversableLike$class.map(TraversableLike.sca=
la:234)
>         at scala.collection.AbstractTraversable.map(Traversable.scala:104=
)
>         at kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.sc=
ala:392)
>         at kafka.server.ReplicaManager.appendMessages(ReplicaManager.scal=
a:328)
>         at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:4=
05)
>         at kafka.server.KafkaApis.handle(KafkaApis.scala:76)
>         at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala=
:60)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: failed to read chunk
>         at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputSt=
ream.java:433)
>         at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.jav=
a:167)
>         at java.io.DataInputStream.readFully(DataInputStream.java:195)
>         at java.io.DataInputStream.readLong(DataInputStream.java:416)
>         at kafka.message.ByteBufferMessageSet$$anon$1.readMessageFromStre=
am(ByteBufferMessageSet.scala:118)
>         at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBuffer=
MessageSet.scala:153)
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)