kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sriram Subramanian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5151) Refactor TransactionCoordinator in-memory structure and error handling logic
Date Mon, 15 May 2017 21:44:04 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011385#comment-16011385

Sriram Subramanian commented on KAFKA-5151:

[~guozhang] can we close this?

> Refactor TransactionCoordinator in-memory structure and error handling logic
> ----------------------------------------------------------------------------
>                 Key: KAFKA-5151
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5151
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: core
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
> Current status:
> 1. we are having two types of threads: request handling thread for any client requests
as well as controller requests for `immigration` and `emigration`, and the marker sender thread
for draining queued marker entries and handle responses. They maintain different in-memory
cache structures like the `txnMetadataCache`, and the `pendingTxnMap` which are storing the
same info, and they access some of the shared structures concurrently, like the markers queue
and the markerPurgatory.
> 2. we are having one queue per broker today, and due to the emigration purpose we probably
are having one queue per brokerId + TxnLogPartitionId + DataPartitionId, which would result
in a lot of queues to handle.
> This ticket is for collapsing some of these structures and simplify the access of them
from concurrent threads.

This message was sent by Atlassian JIRA

View raw message