kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurva Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5317) Update KIP-98 to reflect changes during implementation.
Date Tue, 13 Jun 2017 23:22:01 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048515#comment-16048515
] 

Apurva Mehta commented on KAFKA-5317:
-------------------------------------

I updated both the wiki and the google doc to reflect the changes on this list. Closing this
out.

> Update KIP-98 to reflect changes during implementation.
> -------------------------------------------------------
>
>                 Key: KAFKA-5317
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5317
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: clients, core, producer 
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
>            Priority: Blocker
>              Labels: documentation, exactly-once
>             Fix For: 0.11.0.0
>
>
> While implementing the EOS design, there are some minor (or major?) tweaks we made as
we hands-on the code bases. We will compile all these changes in a single run at the end of
the code-ready and this JIRA is for keeping track of all the changes we made.
> 02/27/2017: collapse the two types of transactional log messages into a single type with
key as transactional id, and value as [pid, epoch, transaction_timeout, transaction_state,
[topic partition ] ]. Also using a single memory map in cache instead of two on the TC.
> 03/01/2017: for pid expiration, we decided to use min(transactional id expiration timeout,
topic retention). For topics enabled for compaction only, we just use the transactional timeout.
If the retention setting is larger than the transactional id expiration timeout, then the
pid will be "logically" expired (i.e. we will remove it from the cached pid mapping and ignore
it when rebuilding the cache from the log)
> 03/20/2017: add a new exception type in `o.a.k.common.errors` for invalid transaction
timeout values.
> 03/25/2017: extend WriteTxnMarkerRequest to contain multiple markers for multiple PIDs
with a single request.
> 04/20/2017: add transactionStartTime to TransactionMetadata
> 04/26/2017: added a new retriable error: Errors.CONCURRENT_TRANSACTIONS
> 04/01/2017: We also enforce acks=all on the client when idempotence is enabled. Without
this, we cannot again guarantee idemptoence.
> 04/10/2017: WE also don't pass the underlying exception to `RetriableOffsetCommitFailedException`
anymore: https://issues.apache.org/jira/browse/KAFKA-5052



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message