flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink Kafka connector08 not updating the offsets with the zookeeper
Date Wed, 12 Oct 2016 09:42:32 GMT
Hi Anchit,

Can you re-run your job with the debug level for Flink set to DEBUG?
Then, you should see the following log message every time the offset is
committed of Zookeeper:

"Committing offsets to Kafka/ZooKeeper for checkpoint"

Alternatively, can you check whether the offsets are available in Zookeeper
once the first checkpoint completed?
They should be located in

/consumers/<group.id>/offsets/<topic>/<partition>

In Flink 1.2-SNAPSHOT, Flink is exposing the current offset as a metric for
all kafka connector versions.

Regards,
Robert


On Wed, Oct 12, 2016 at 2:35 AM, Anchit Jatana <development.anchit@gmail.com
> wrote:

> Hi All,
>
> I'm using Flink Kafka connector08. I need to check/monitor the offsets of
> the my flink application's kafka consumer.
>
> When running this:
>
> bin/kafka-consumer-groups.sh --zookeeper <zookeeper-url:port> --describe
> --group <group-id>
>
> I get the message: No topic available for consumer group provided. Why is
> the consumer not updating the offsets with the zookeeper ?
>
> PS: I have enabled checkpointing. Is there any configuration that I'm
> missing or is this some sort of a bug?
>
> Using Flink version 1.1.2
>
> Thank you
>
> Regards,
> Anchit
>

Mime
View raw message