kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gwen Shapira (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5317) Update KIP-98 to reflect changes during implementation.
Date Thu, 08 Jun 2017 22:25:18 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043541#comment-16043541

Gwen Shapira commented on KAFKA-5317:

I understand the importance, but is it really a release blocker?

> Update KIP-98 to reflect changes during implementation.
> -------------------------------------------------------
>                 Key: KAFKA-5317
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5317
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: clients, core, producer 
>            Reporter: Apurva Mehta
>            Priority: Blocker
>              Labels: documentation, exactly-once
>             Fix For:
> While implementing the EOS design, there are some minor (or major?) tweaks we made as
we hands-on the code bases. We will compile all these changes in a single run at the end of
the code-ready and this JIRA is for keeping track of all the changes we made.
> 02/27/2017: collapse the two types of transactional log messages into a single type with
key as transactional id, and value as [pid, epoch, transaction_timeout, transaction_state,
[topic partition ] ]. Also using a single memory map in cache instead of two on the TC.
> 03/01/2017: for pid expiration, we decided to use min(transactional id expiration timeout,
topic retention). For topics enabled for compaction only, we just use the transactional timeout.
If the retention setting is larger than the transactional id expiration timeout, then the
pid will be "logically" expired (i.e. we will remove it from the cached pid mapping and ignore
it when rebuilding the cache from the log)
> 03/20/2017: add a new exception type in `o.a.k.common.errors` for invalid transaction
timeout values.
> 03/25/2017: extend WriteTxnMarkerRequest to contain multiple markers for multiple PIDs
with a single request.
> 04/20/2017: add transactionStartTime to TransactionMetadata
> 04/26/2017: added a new retriable error: Errors.CONCURRENT_TRANSACTIONS
> 04/01/2017: We also enforce acks=all on the client when idempotence is enabled. Without
this, we cannot again guarantee idemptoence.
> 04/10/2017: WE also don't pass the underlying exception to `RetriableOffsetCommitFailedException`
anymore: https://issues.apache.org/jira/browse/KAFKA-5052

This message was sent by Atlassian JIRA

View raw message