kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject Re: [DISCUSS] KIP 88: OffsetFetch Protocol Update
Date Thu, 15 Dec 2016 23:02:52 GMT
Hi all,

Even though KIP-88 was recently approved, due to a limitation that comes 
with the proposed protocol change in KIP-88 I'll have to re-open it to 
address the problem.
I'd like to thank Jason Gustafson for catching this issue.

I'll explain this in the KIP as well, but to summarize, KIP-88 suggests 
adding the option of passing a "null" array in FetchOffset request to 
query all existing offsets for a consumer group. It does not suggest any 
modification to FetchOffset response.

In the existing protocol, group or coordinator related errors are reported 
along with each partition in the OffsetFetch response.

If there are partitions in the request, they are guaranteed to appear in 
the response (there could be an error code associated with each). So if 
there is an error, it is reported back by being attached to some partition 
in the request.
If an empty array is passed, no error is reported (no matter what the 
group or coordinator status is). The response comes back with an empty 
list.

With the proposed change in KIP-88 we could have a scenario in which a 
null array is sent in FetchOffset request, and due to some errors (for 
example if coordinator just started and hasn't caught up yet with the 
offset topic), an empty list is returned in the FetchOffset response (the 
group may or may not actually be empty). The issue is in situations like 
this no error can be returned in the response because there is no 
partition to attach the error to.

I'll update the KIP with more details and propose to add to OffsetFetch 
response schema an "error_code" at the top level that can be used to 
report group related errors (instead of reporting those errors with each 
individual partition).

I apologize if this causes any inconvenience.

Feedback and comments are always welcome.

Thanks.
--Vahid


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message