kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie Wang <jamie.w...@actuate.com>
Subject How do you keep track of offset in a partition
Date Tue, 29 Jan 2013 01:05:41 GMT
Hi,

We are using 0.72 version of Kafka on Windows. I am wondering what is the right way to fetch
data and keep track of offset in a partition. For example, I am currently assuming the first
message the producer sent to the broker is at offset 0. So far it seems working. Is this correct
assumption?

Let' say 2 days later, the first 100 messages on the broker is discarded because it passed
retention.hours set in the config file. Now what is the offset I should use to retrieve the
first message in the partition?  And let's also say the offset I had for the 80th message
is now not valid.
What is the right way to get the correct offset to fetch in the consumer?

What is the purpose of the api for getting a list of valid offsets for all segments in a partition?

Thanks in advance for your help.

Jamie

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message