kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-4102) Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
Date Tue, 03 Jan 2017 23:08:58 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ewen Cheslack-Postava resolved KAFKA-4102.
------------------------------------------
    Resolution: Won't Fix

[yf] Currently the Java clients are only compatible with brokers that are the same version
or newer. Since MM uses the clients, it has this same limitation.

This won't be fixed for older versions of Kafka, but https://cwiki.apache.org/confluence/display/KAFKA/KIP-97%3A+Improved+Kafka+Client+RPC+Compatibility+Policy
will be improving compatibility for 0.10+ brokers & clients.

> Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
> -----------------------------------------------------------------
>
>                 Key: KAFKA-4102
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4102
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: yf
>             Fix For: 0.10.2.0
>
>
> I want this feature (https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement)
in kafka new version.
> But it seems can't support kafka 0.8.X ?
> Cmd: ./kafka_2.11-0.10.0.1/bin/kafka-mirror-maker.sh --producer.config svc_run/mirror_kafka_write_online2streaming/producer.properties
--consumer.config svc_run/mirror_kafka_write_online2streaming/consumer.properties --whitelist
sandbox
> Producer config:
> queue.buffering.max.messages=16384
> bootstrap.servers=XXX
> send.buffer.bytes=131072
> message.send.max.retries=1048576
> consumer config:
> zookeeper.connect=XXXX
> group.id=test_mirror
> consumer.timeout.ms=-1
> zookeeper.connection.timeout.ms=60000
> zookeeper.session.timeout.ms=60000
> socket.receive.buffer.bytes=-1
> auto.commit.interval.ms=5000
> auto.commit.enable=false
> But I got failures like follows:
> FetchRequest@33437902 (kafka.consumer.ConsumerFetcherThread)
> java.lang.IllegalArgumentException
> 	at java.nio.Buffer.limit(Buffer.java:275)
> 	at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
> 	at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
> 	at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> 	at scala.collection.immutable.Range.foreach(Range.scala:160)
> 	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> 	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> 	at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
> 	at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
> 	at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
> 	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> 	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> 	at scala.collection.immutable.Range.foreach(Range.scala:160)
> 	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> 	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> 	at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
> 	at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
> 	at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
> 	at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
> 	at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
> 	at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
> 	at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message