kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Damick <jeffreydam...@gmail.com>
Subject Re: kafka-49
Date Thu, 21 Jul 2011 14:03:44 GMT
But if there are no replicas, a produce with an ACK requested would at least
force a flush to disk I assume?  That would be enough for me right now.
And I agree with Jay's comment, the tradeoff of durability vs. throughput
could be chosen by the producer, either with a different message type or
flag in the request.

As for the offset of the message, I'd really like to be able to know what it
is after I produce so that it can be logged and monitored.  Then if needed
something or someone can quickly grab the latest message produced on a topic
for metrics, monitoring, and debugging if something goes awry, otherwise I
need to have the producer also consume the topic to figure it out.. (the
data I'm transporting is a little more important than logs)

So back to the ACK'ing, I'd willing to submit a patch for at least the
optional flag or new message type to force a flush to disk if it would
help..

thanks,
-jeff


On Wed, Jul 20, 2011 at 8:51 PM, Jun Rao <junrao@gmail.com> wrote:

> Jeff,
>
> I was thinking of making the ACK mandatory for the producer. The ACK can be
> sent when the message either hits 1 replicas or multiple replicas,
> depending
> on the setting.
>
> Having the ACK include the starting offset of the message seems reasonable.
> It will be a bit complicated for multisend since multiple offsets have to
> be
> returned. What do you need the offset for?
>
> Thanks,
>
> Jun
>
> On Wed, Jul 20, 2011 at 12:47 PM, Jeffrey Damick <jeffreydamick@gmail.com
> >wrote:
>
> > Is there any current thought around KAFKA-49, for acknowledgement of
> > producers?
> > Will this be optional, a new message type(s)?
> > Will the ack be synchronous or asynchronous or depending on request type?
> >
> > It would be fantastic if the ack contained the starting offset of the
> > message published, and not just the ending.
> >
> > This is quickly becoming an issue for us, so we may be able to provide
> some
> > help in this area..
> >
> >
> > thanks
> > -jeff
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message