Mailing-List: contact kafka-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: kafka-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of jeffreydamick@gmail.com
 designates 209.85.212.47 as permitted sender)
References: 
 <CAAg6m_bpa=8nO56ES2Qg5tZ4Ck7zdKqm4zJQ2gp_-YVu=gVvaA@mail.gmail.com>
 <CAFbh0Q1wuqcA833xn_=cVAkaOtFKG5kd5tNtTVS7zCQfVvpGRQ@mail.gmail.com>
 <CAOeJiJi7Kog5zCjJnVHb_Fr0A4aK4Gv7eJ4bSv=R9gFBSXVqQA@mail.gmail.com>
 <CAFbh0Q2q=DZiRbLow0PW098k-zLP9JbSm3hy9Zz+ZB3DbXawVA@mail.gmail.com>
 <CAOeJiJiNBJFDtZ-chicx3vxogysCb0Jgj9YbwW__1KV_EuqOGw@mail.gmail.com>
 <CAFbh0Q1k0rmRmqBdHd4YdXynX6=Rvm89SbdL_nVGbw6kBUEZTw@mail.gmail.com>
In-Reply-To: 
 <CAFbh0Q1k0rmRmqBdHd4YdXynX6=Rvm89SbdL_nVGbw6kBUEZTw@mail.gmail.com>
Mime-Version: 1.0 (iPhone Mail 8C148)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii
Message-Id: <68B1D633-3BE4-46D2-9158-55DF2D1991EF@gmail.com>
Cc: "kafka-dev@incubator.apache.org" <kafka-dev@incubator.apache.org>
From: Jeffrey Damick <jeffreydamick@gmail.com>
Subject: Re: kafka-49
Date: Thu, 21 Jul 2011 21:08:03 -0400
To: "kafka-dev@incubator.apache.org" <kafka-dev@incubator.apache.org>

That would seem sub-optimal.. Perhaps you could have a separate vip for prod=
ucing that was configured by some process watching zk to determine which is l=
eader or some custom health check maybe via restful api..just a thought.

Sent from my iPhone

On Jul 21, 2011, at 6:33 PM, Jun Rao <junrao@gmail.com> wrote:

> If we have ACK, the producer can catch any exception and resend.
>=20
> Jun
>=20
> On Thu, Jul 21, 2011 at 12:47 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
>=20
>> Yeah, that makes sense. How are we going to handle production through a v=
ip
>> in that case?
>>=20
>> -Jay
>>=20
>> On Thu, Jul 21, 2011 at 7:44 AM, Jun Rao <junrao@gmail.com> wrote:
>>=20
>>> Without replication, we can make ACK optional. With replication, a
>> producer
>>> can only write to the leader replica. Without ACK, there is no way that
>> the
>>> broker can inform the producer that it's trying to write to the wrong
>>> broker.
>>>=20
>>> Jun
>>>=20
>>> On Wed, Jul 20, 2011 at 8:58 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
>>>=20
>>>> It might be nice to consider making the ack optional and part of the
>>>> request. The current behavior is good for many uses, the request is
>>>> instantaneously written to the socket buffer but sent asynchronously. I=

>>>> think that is a valuable use case where you care about throughput. I
>>> guess
>>>> the question is whether the asynchronous api already covers that well
>>>> enough
>>>> and how much complexity exposing that causes.
>>>>=20
>>>> -Jay
>>>>=20
>>>> On Wed, Jul 20, 2011 at 5:51 PM, Jun Rao <junrao@gmail.com> wrote:
>>>>=20
>>>>> Jeff,
>>>>>=20
>>>>> I was thinking of making the ACK mandatory for the producer. The ACK
>>> can
>>>> be
>>>>> sent when the message either hits 1 replicas or multiple replicas,
>>>>> depending
>>>>> on the setting.
>>>>>=20
>>>>> Having the ACK include the starting offset of the message seems
>>>> reasonable.
>>>>> It will be a bit complicated for multisend since multiple offsets
>> have
>>> to
>>>>> be
>>>>> returned. What do you need the offset for?
>>>>>=20
>>>>> Thanks,
>>>>>=20
>>>>> Jun
>>>>>=20
>>>>> On Wed, Jul 20, 2011 at 12:47 PM, Jeffrey Damick <
>>>> jeffreydamick@gmail.com
>>>>>> wrote:
>>>>>=20
>>>>>> Is there any current thought around KAFKA-49, for acknowledgement
>> of
>>>>>> producers?
>>>>>> Will this be optional, a new message type(s)?
>>>>>> Will the ack be synchronous or asynchronous or depending on request
>>>> type?
>>>>>>=20
>>>>>> It would be fantastic if the ack contained the starting offset of
>> the
>>>>>> message published, and not just the ending.
>>>>>>=20
>>>>>> This is quickly becoming an issue for us, so we may be able to
>>> provide
>>>>> some
>>>>>> help in this area..
>>>>>>=20
>>>>>>=20
>>>>>> thanks
>>>>>> -jeff
>>>>>>=20
>>>>>=20
>>>>=20
>>>=20
>>=20