kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-1430) Purgatory redesign
Date Wed, 16 Jul 2014 23:49:05 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Guozhang Wang updated KAFKA-1430:

    Attachment: KAFKA-1430.patch

> Purgatory redesign
> ------------------
>                 Key: KAFKA-1430
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1430
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 0.8.2
>            Reporter: Jun Rao
>            Assignee: Guozhang Wang
>         Attachments: KAFKA-1430.patch, KAFKA-1430.patch, KAFKA-1430.patch, KAFKA-1430_2014-06-05_17:07:37.patch,
KAFKA-1430_2014-06-05_17:13:13.patch, KAFKA-1430_2014-06-05_17:40:53.patch, KAFKA-1430_2014-06-10_11:22:06.patch,
KAFKA-1430_2014-06-10_11:26:02.patch, KAFKA-1430_2014-07-11_10:59:13.patch
> We have seen 2 main issues with the Purgatory.
> 1. There is no atomic checkAndWatch functionality. So, a client typically first checks
whether a request is satisfied or not and then register the watcher. However, by the time
the watcher is registered, the registered item could already be satisfied. This item won't
be satisfied until the next update happens or the delayed time expires, which means the watched
item could be delayed. 
> 2. FetchRequestPurgatory doesn't quite work. This is because the current design tries
to incrementally maintain the accumulated bytes ready for fetch. However, this is difficult
since the right time to check whether a fetch (for regular consumer) request is satisfied
is when the high watermark moves. At that point, it's hard to figure out how many bytes we
should incrementally add to each pending fetch request.
> The problem has been reported in KAFKA-1150 and KAFKA-703.

This message was sent by Atlassian JIRA

View raw message