kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ran RanUser <ranuse...@gmail.com>
Subject SimpleConsumer - Should FetchRequests span log segments?
Date Sat, 29 Jun 2013 04:53:28 GMT
Using Kafka 0.8, when specifying a starting offset and appropriate
fetchSize, SimpleConsumer will only return up to the highest offset of the
log segment containing the starting offset.

For example,

log segment #1 contains offsets 1 - 10
log segment #2 contiains offsets 11 - 100

A fetch request specifying offset 9, will only return offsets 9 and 10.
Specifying starting offset 11 will return 11 - 100

Must one check fetchResponse.highwatermark each time to determine if there
are higher numbered offsets, and continue submitting fetchRequests ?

I assumed SimpleConsumer would return offsets/messages spanning log
segments, up to the specified fetchSize.


Thanks!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message