kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Damick <jeffreydam...@gmail.com>
Subject Re: kafka-49
Date Fri, 22 Jul 2011 01:08:03 GMT
That would seem sub-optimal.. Perhaps you could have a separate vip for producing that was
configured by some process watching zk to determine which is leader or some custom health
check maybe via restful api..just a thought.

Sent from my iPhone

On Jul 21, 2011, at 6:33 PM, Jun Rao <junrao@gmail.com> wrote:

> If we have ACK, the producer can catch any exception and resend.
> 
> Jun
> 
> On Thu, Jul 21, 2011 at 12:47 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
> 
>> Yeah, that makes sense. How are we going to handle production through a vip
>> in that case?
>> 
>> -Jay
>> 
>> On Thu, Jul 21, 2011 at 7:44 AM, Jun Rao <junrao@gmail.com> wrote:
>> 
>>> Without replication, we can make ACK optional. With replication, a
>> producer
>>> can only write to the leader replica. Without ACK, there is no way that
>> the
>>> broker can inform the producer that it's trying to write to the wrong
>>> broker.
>>> 
>>> Jun
>>> 
>>> On Wed, Jul 20, 2011 at 8:58 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
>>> 
>>>> It might be nice to consider making the ack optional and part of the
>>>> request. The current behavior is good for many uses, the request is
>>>> instantaneously written to the socket buffer but sent asynchronously. I
>>>> think that is a valuable use case where you care about throughput. I
>>> guess
>>>> the question is whether the asynchronous api already covers that well
>>>> enough
>>>> and how much complexity exposing that causes.
>>>> 
>>>> -Jay
>>>> 
>>>> On Wed, Jul 20, 2011 at 5:51 PM, Jun Rao <junrao@gmail.com> wrote:
>>>> 
>>>>> Jeff,
>>>>> 
>>>>> I was thinking of making the ACK mandatory for the producer. The ACK
>>> can
>>>> be
>>>>> sent when the message either hits 1 replicas or multiple replicas,
>>>>> depending
>>>>> on the setting.
>>>>> 
>>>>> Having the ACK include the starting offset of the message seems
>>>> reasonable.
>>>>> It will be a bit complicated for multisend since multiple offsets
>> have
>>> to
>>>>> be
>>>>> returned. What do you need the offset for?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Jun
>>>>> 
>>>>> On Wed, Jul 20, 2011 at 12:47 PM, Jeffrey Damick <
>>>> jeffreydamick@gmail.com
>>>>>> wrote:
>>>>> 
>>>>>> Is there any current thought around KAFKA-49, for acknowledgement
>> of
>>>>>> producers?
>>>>>> Will this be optional, a new message type(s)?
>>>>>> Will the ack be synchronous or asynchronous or depending on request
>>>> type?
>>>>>> 
>>>>>> It would be fantastic if the ack contained the starting offset of
>> the
>>>>>> message published, and not just the ending.
>>>>>> 
>>>>>> This is quickly becoming an issue for us, so we may be able to
>>> provide
>>>>> some
>>>>>> help in this area..
>>>>>> 
>>>>>> 
>>>>>> thanks
>>>>>> -jeff
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Mime
View raw message