apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siyuan Hua <siy...@datatorrent.com>
Subject Re: Question on KafkaInputOperator - with kafka 0.8.2.1
Date Wed, 06 Jul 2016 18:30:35 GMT
Thomas,

I think a if condition check should be able to make this util class support
both 0.8.1.* and 0.8.2.*

Regards,
Siyuan

On Tue, Jul 5, 2016 at 2:22 PM, Thomas Weise <thomas.weise@gmail.com> wrote:

> We have had this question often.
>
> I think it would be good to bump the version in malhar-contrib as most
> users are on 0.8.2?
>
> ---------- Forwarded message ----------
> From: Thomas Weise <thomas.weise@gmail.com>
> Date: Tue, Jul 5, 2016 at 2:20 PM
> Subject: Re: Question on KafkaInputOperator - with kafka 0.8.2.1
> To: users@apex.apache.org
>
>
> Hi Eric,
>
> You need to use the 0.8.1 version of the client library. It can talk to the
> 0.8.2 cluster.
>
> Thanks,
> Thomas
>
>
> On Tue, Jul 5, 2016 at 2:16 PM, Martin, Eric <Eric.Martin2@capitalone.com>
> wrote:
>
> > Hi all,
> >
> >
> >
> > I am using the 0.8.x version of the Kafka Input Operators. When using
> > these operators, I am able to run the application fine while using kafka
> > version 0.8.1.1. However, if I use version 0.8.2.x, I am not able to run
> > the application even though the kafka cluster that I am connecting is
> > running on 0.8.2.1.
> >
> >
> >
> > The exception that I am getting is:
> >
> > *java.lang.NoSuchMethodError:
> > kafka.cluster.Broker.getConnectionString()Ljava/lang/String;*
> >
> >
> >
> > *at
> >
> com.datatorrent.contrib.kafka.KafkaMetadataUtil.getBrokers(KafkaMetadataUtil.java:114)*
> >
> > *at
> >
> com.datatorrent.contrib.kafka.KafkaConsumer.initBrokers(KafkaConsumer.java:131)*
> >
> > *at
> >
> com.datatorrent.contrib.kafka.AbstractKafkaInputOperator.definePartitions(AbstractKafkaInputOperator.java:488)*
> >
> >
> >
> > It would appear that the getConnectionString() method is not supported in
> > kafka 0.8.2.x. Using 0.8.1.x is not a huge issue for most things, but I
> am
> > looking to keep track of offsets in topics and this only appears to be
> > available with kafka 0.8.2.x.
> >
> >
> >
> > Is there any known work around for this issue, so that I can use 0.8.2.x
> > in the application?
> >
> >
> >
> > Thanks,
> >
> > Eric
> >
> > ------------------------------
> >
> > The information contained in this e-mail is confidential and/or
> > proprietary to Capital One and/or its affiliates and may only be used
> > solely in performance of work or services for Capital One. The
> information
> > transmitted herewith is intended only for use by the individual or entity
> > to which it is addressed. If the reader of this message is not the
> intended
> > recipient, you are hereby notified that any review, retransmission,
> > dissemination, distribution, copying or other use of, or taking of any
> > action in reliance upon this information is strictly prohibited. If you
> > have received this communication in error, please contact the sender and
> > delete the material from your computer.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message