kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4102) Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
Date Tue, 03 Jan 2017 23:08:58 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ewen Cheslack-Postava updated KAFKA-4102:
-----------------------------------------
    Fix Version/s:     (was: 0.10.2.0)

> Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
> -----------------------------------------------------------------
>
>                 Key: KAFKA-4102
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4102
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: yf
>
> I want this feature (https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement)
in kafka new version.
> But it seems can't support kafka 0.8.X ?
> Cmd: ./kafka_2.11-0.10.0.1/bin/kafka-mirror-maker.sh --producer.config svc_run/mirror_kafka_write_online2streaming/producer.properties
--consumer.config svc_run/mirror_kafka_write_online2streaming/consumer.properties --whitelist
sandbox
> Producer config:
> queue.buffering.max.messages=16384
> bootstrap.servers=XXX
> send.buffer.bytes=131072
> message.send.max.retries=1048576
> consumer config:
> zookeeper.connect=XXXX
> group.id=test_mirror
> consumer.timeout.ms=-1
> zookeeper.connection.timeout.ms=60000
> zookeeper.session.timeout.ms=60000
> socket.receive.buffer.bytes=-1
> auto.commit.interval.ms=5000
> auto.commit.enable=false
> But I got failures like follows:
> FetchRequest@33437902 (kafka.consumer.ConsumerFetcherThread)
> java.lang.IllegalArgumentException
> 	at java.nio.Buffer.limit(Buffer.java:275)
> 	at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
> 	at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
> 	at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> 	at scala.collection.immutable.Range.foreach(Range.scala:160)
> 	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> 	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> 	at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
> 	at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
> 	at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
> 	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> 	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> 	at scala.collection.immutable.Range.foreach(Range.scala:160)
> 	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> 	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> 	at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
> 	at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
> 	at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
> 	at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
> 	at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
> 	at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
> 	at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message