Mailing-List: contact users-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@apex.apache.org
MIME-Version: 1.0
In-Reply-To: <0614D484-1FA4-46B8-8567-29B36D4A4EF6@target.com>
References: <0614D484-1FA4-46B8-8567-29B36D4A4EF6@target.com>
From: Thomas Weise <thomas.weise@gmail.com>
Date: Mon, 6 Jun 2016 15:23:25 -0700
Message-ID: <CA+5xAo06RjEqN2UyvzC1CQ_dnXaja_F6mNmY1ixg2pP_fNxBCg@mail.gmail.com>
Subject: Re: kafka offset commit
To: users@apex.apache.org
Content-Type: multipart/alternative; boundary=001a114592ac692ed50534a3852b
archived-at: Mon, 06 Jun 2016 22:23:31 -0000

--001a114592ac692ed50534a3852b
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Raja,

Which Kafka version are you using?

With the new 0.9 connector there is no need for the offset manager:

https://github.com/apache/apex-malhar/tree/master/kafka/src/main/java/org/a=
pache/apex/malhar/kafka

Thanks,
Thomas


On Mon, Jun 6, 2016 at 3:06 PM, Raja.Aravapalli <Raja.Aravapalli@target.com=
>
wrote:

> Hi
>
> Can someone please help me understand, where will the offsets be stored
> when consuming with =E2=80=9C*KafkaSinglePortStringInputOperator*=E2=80=
=9D  ?
>
> And, how to handle restarts ?
>
>
> I worked with Storm earlier, Storm maintains the offsets in zookeeper and
> client id is maintained for every consumer, using which
>
> - we can see what is the current offset status for a given partition &
> modify them as well using zookeeper-cli !!
> - restarts can be handled
>
>
> As per the Apex documentation, I can see, that using OffsetManager we can
> handle the restarts effectively, but couldn=E2=80=99t find any examples t=
o refer=E2=80=A6
>
> How clientId can be used to retrieve offsets status
> And ability to edit the offsets etc
>
> can someone pls help me find this ?
>
>
> Thanks a lot!!
>
>
> -Regards,
> Raja.
>
>
>
>

--001a114592ac692ed50534a3852b
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Raja,<div><br></div><div>Which Kafka version are you us=
ing?</div><div><br></div><div>With the new 0.9 connector there is no need f=
or the offset manager:</div><div><br></div><div><a href=3D"https://github.c=
om/apache/apex-malhar/tree/master/kafka/src/main/java/org/apache/apex/malha=
r/kafka">https://github.com/apache/apex-malhar/tree/master/kafka/src/main/j=
ava/org/apache/apex/malhar/kafka</a><br></div><div><br></div><div>Thanks,</=
div><div>Thomas</div><div><br></div></div><div class=3D"gmail_extra"><br><d=
iv class=3D"gmail_quote">On Mon, Jun 6, 2016 at 3:06 PM, Raja.Aravapalli <s=
pan dir=3D"ltr">&lt;<a href=3D"mailto:Raja.Aravapalli@target.com" target=3D=
"_blank">Raja.Aravapalli@target.com</a>&gt;</span> wrote:<br><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p=
adding-left:1ex">


<div style=3D"word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-fam=
ily:Calibri,sans-serif">
<div>Hi=C2=A0</div>
<div><br>
</div>
<div>Can someone please help me understand, where will the offsets be store=
d when consuming with =E2=80=9C<b>KafkaSinglePortStringInputOperator</b>=E2=
=80=9D =C2=A0?</div>
<div><br>
</div>
<div>And, how to handle restarts ?</div>
<div><br>
</div>
<div><br>
</div>
<div>I worked with Storm earlier, Storm maintains the offsets in zookeeper =
and client id is maintained for every consumer, using which=C2=A0</div>
<div><br>
</div>
<div>- we can see what is the current offset status for a given partition &=
amp; modify them as well using zookeeper-cli !!</div>
<div>- restarts can be handled</div>
<div><br>
</div>
<div><br>
</div>
<div>As per the Apex documentation, I can see, that using OffsetManager we =
can handle the restarts effectively, but couldn=E2=80=99t find any examples=
 to refer=E2=80=A6=C2=A0</div>
<div><br>
</div>
<div>How clientId can be used to retrieve offsets status</div>
<div>And ability to edit the offsets etc</div>
<div><br>
</div>
<div>can someone pls help me find this ?=C2=A0</div>
<div><br>
</div>
<div><br>
</div>
<div>Thanks a lot!!</div>
<div><br>
</div>
<div><br>
</div>
<div>-Regards,</div>
<div>Raja.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div></div>
</div>
</div>

</blockquote></div><br></div>

--001a114592ac692ed50534a3852b--