kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saher Ahwal <saher.ah...@gmail.com>
Subject Re: Kafka Exactly-Once Semantics in .NET support
Date Wed, 01 Jul 2020 16:59:13 GMT
Hi Boyang

Thank you so much for the reply. I opened an issue on confluent dotnet on
github and it was answered here:
https://github.com/confluentinc/confluent-kafka-dotnet/issues/1342
I will re-paste it here for the benefit of others who might be interested:

Matt's reply from Confluent:
"

We don't have specific exceptions (yet) - rather you'll get a
KafkaException with
an Error field set. Error has properties that indicate whether the error is
retriable etc. and an ErrorCode to tell you specifically what the error is.

We do intend to add more specific exception types in the future (its better
UX), but we don't have them presently.

We will hopefully document this better when we implement KIP-447 (currently
not implemented).

https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics

this example might be helpful:
https://github.com/confluentinc/confluent-kafka-dotnet/tree/master/examples/Transactions

"

Thanks
Saher



On Tue, Jun 30, 2020 at 11:43 PM Boyang Chen <reluctanthero104@gmail.com>
wrote:

> Thanks for the interest Saher. TBH, I don't know how the .NET confluent
> library works, but I would assume it is based on the
> https://github.com/edenhill/librdkafka. Maybe propose an issue there as
> well to attract some expertes.
>
> Boyang
>
> On Tue, Jun 30, 2020 at 3:22 PM Saher Ahwal <saher.ahwal@gmail.com> wrote:
>
> > Hi
> >
> > I am working on exactly-once semantics with Kafka and I have streaming
> > scenario of read-process-write. I noticed the new exactly-once
> scalability
> > design with correctness in case of partition reassignment here:
> >
> >
> https://docs.google.com/document/d/1LhzHGeX7_Lay4xvrEXxfciuDWATjpUXQhrEIkph9qRE/edit#
> >
> > It all looks great and I understand we need to rely on
> > PendingTransatcionException and ProducerFencedException. However, we are
> > using confluent library and .NET and I don't see any of these exceptions.
> >
> > How do I ensure correctness of fencing of zombie producer transactions
> when
> > using .NET confluent library? What exceptions are retry-able and what are
> > not? Can I call abortTransaction on any Kafka exception ? I don't find
> good
> > examples in the documentation.
> > Any pointers or answers are kindly appreciated.
> >
> > Thanks in advance
> > Saher
> >
> >
> >
> > --
> > *Saher Ahwal*
> >
> > *Massachusetts Institute of Technology '13, '14*
> > *Department of Electrical Engineering and Computer Science*
> > *saher@alum.mit.edu <saher@mit.edu> | 617 680 4877*
> >
>


-- 
*Saher Ahwal*

*Massachusetts Institute of Technology '13, '14*
*Department of Electrical Engineering and Computer Science*
*saher@alum.mit.edu <saher@mit.edu> | 617 680 4877*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message