kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Andre Pearce (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-5537) Subscribe Earliest is not working as in 0.10.2.1
Date Sat, 08 Jul 2017 08:34:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079044#comment-16079044
] 

Michael Andre Pearce edited comment on KAFKA-5537 at 7/8/17 8:33 AM:
---------------------------------------------------------------------

[~rsivaram] yes this does seem to resolve this issue. Thanks.

I guess now the question is whilst this all works debugging these individual cases, whats
the definitive rule (calculation one should perform based on rebalance ms set and the number
of consumers) so it can be applied to all different scenarios in test suites etc people will
have. Im fairly sure you don't want to be debugging all our test cases, but still this seems
to be a bit of a dark art / trial and error.

As earlier you inidcated the time for the rebalance was 3000 so time to wait should be that
+ 1000 = 4000 millis but now we have to wait 8000 millis in this particular case.

Also from a system monitoring point of view the exact same question applies how does anyone
work out or calculate the time it should take before its safe to assume its rebalanced or
the maximum amount of time expected so if not completed in X seconds one could alert to a
system fault

It also be great to have this information in the documentation.




was (Author: michael.andre.pearce):
[~rsivaram] yes this does seem to resolve this issue. 

I guess now the question is whilst this all works debugging these individual cases, whats
the definitive rule (calculation one should perform based on rebalance ms set and the number
of consumers) so it can be applied to all different scenarios in test suites etc people will
have. Im fairly sure you don't want to be debugging all our test cases, but still this seems
to be a bit of a dark art / trial and error.

As earlier you inidcated the time for the rebalance was 3000 so time to wait should be that
+ 1000 = 4000 millis but now we have to wait 8000 millis in this particular case.

Also from a system monitoring point of view the exact same question applies how does anyone
work out or calculate the time it should take before its safe to assume its rebalanced or
the maximum amount of time expected so if not completed in X seconds one could alert to a
system fault

It also be great to have this information in the documentation.




> Subscribe Earliest is not working as in 0.10.2.1
> ------------------------------------------------
>
>                 Key: KAFKA-5537
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5537
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.11.0.0
>            Reporter: Michael Andre Pearce
>            Priority: Critical
>         Attachments: kafka_0.10.2.1.log, kafka_0.11.0.0.log, KafkaSub.java, KafkaSubLatest.java
>
>
> We have seen issue with subscription where auto offset when set to earliest (and also
latest) does not behave the same as in 0.10.2.1 release.
> We have managed to create a repeatable test for this, which passes when pointing to 0.10.2.1
broker.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message