kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6705) producer.send() should be non-blocking
Date Fri, 23 Mar 2018 03:52:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410765#comment-16410765

Ismael Juma commented on KAFKA-6705:

[~lindong], there's a long-standing Jira about this, it may be worth searching for it. One
reason why the current interface blocks is to provide back-pressure in case you have exhausted
the buffer pool. If `send` is completely non-blocking, then this would no longer be possible.
In any case, feel free to add your thoughts on the original JIRA.

> producer.send() should be non-blocking
> --------------------------------------
>                 Key: KAFKA-6705
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6705
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Dong Lin
>            Assignee: Dong Lin
>            Priority: Major
> Currently producer.send() may block on metadata or full buffer for up to max.block.ms.
This behavior is well documented but it is a bit sub-optimal. Since we return a future we
should be able to make producer.send() completely non-blocking. One idea is to simply insert
the record into a global queue shared across all partitions, and let the sender thread fetch
record from this queue and send to broker.

This message was sent by Atlassian JIRA

View raw message