kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5364) Producer attempts to send transactional messages before adding partitions to transaction
Date Sun, 04 Jun 2017 01:31:04 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036140#comment-16036140
] 

ASF GitHub Bot commented on KAFKA-5364:
---------------------------------------

GitHub user hachikuji opened a pull request:

    https://github.com/apache/kafka/pull/3231

    KAFKA-5364: ensurePartitionAdded does not handle pending partitions in abortable error
state

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/hachikuji/kafka KAFKA-5364

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3231.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3231
    
----
commit 7e65cf7e4642e8dca29a02321e29e4701a1712ba
Author: Jason Gustafson <jason@confluent.io>
Date:   2017-06-04T01:26:51Z

    KAFKA-5364: ensurePartitionAdded does not handle pending partitions in abortable error
state

----


> Producer attempts to send transactional messages before adding partitions to transaction
> ----------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5364
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5364
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: clients, core, producer 
>    Affects Versions: 0.11.0.0
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
>            Priority: Blocker
>              Labels: exactly-once
>             Fix For: 0.11.0.0
>
>         Attachments: KAFKA-5364.tar.gz
>
>
> Due to a race condition between the sender thread and the producer.send(), the following
is possible: 
> # In KakfaProducer.doSend(), we add partitions to the transaction and then do accumulator.append.

> # In Sender.run(), we check whether there are transactional request. If there are, we
send them and wait for the response. 
> # If there aren't we drain the accumulator queue and send the produce requests.
> # The problem is that the sequence step 2, 1, 3 is entire possible. This means that we
won't send the 'AddPartitions' request but yet try to send the produce data. Which results
in a fatal error and requires the producer to close. 
> The solution is that in the accumulator.drain, we should check again if there are pending
add partitions requests, and if so, don't drain anything.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message