kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: kafka replication blog
Date Fri, 08 Feb 2013 15:59:45 GMT
That's right. If you are partitioning by key, that means you insist a
message has to go to a certain partition, whether it's available or not.
So, if a partition is not available, we will drop the message for the
partition in the async mode and consistently throw an exception to the
caller in the sync mode.

Thanks,

Jun

On Fri, Feb 8, 2013 at 1:41 AM, Michal Haris <michal.haris@visualdna.com>wrote:

> So if the produces are partitioning by key we have to have replication if
> we dont want messages to get lost when partition goes down l right ?
> Thanks
> On Feb 8, 2013 5:12 AM, "Jun Rao" <junrao@gmail.com> wrote:
>
> > We have fixed this issue in 0.8. Withreplication factor 1, if the
> producer
> > doesn't care about partitioning by key, messages will be sent to
> partitions
> > that are currently available.
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Feb 7, 2013 at 3:11 PM, Michal Haris <michal.haris@visualdna.com
> > >wrote:
> >
> > > Same here, summary was need as we have a fairly large ecosystem of
> > multiple
> > > 0.7.2 clusters and I am planning to test upgrade to 0.8.
> > > However, one thing  creeping at the back of my mind regarding 0.8 is
> > > something i have spotted in one thread few weeks ago namely that the
> > > rebalance behaviour of producers is not as robust as in 0.7.x without
> > > replication and i remeber there was no designed solution at the time -
> > any
> > > news here ? Basically our usecase doesn't require replication but
> logical
> > > offsets and some other things introduced would solve some problems.
> > > On Feb 7, 2013 7:11 PM, "Vaibhav Puranik" <vpuranik@gmail.com> wrote:
> > >
> > > > Same here. Thanks a lot Jun.
> > > >
> > > > Regards,
> > > > Vaibhav
> > > >
> > > > On Thu, Feb 7, 2013 at 10:38 AM, Felix GV <felix@mate1inc.com>
> wrote:
> > > >
> > > > > Thanks Jun!
> > > > >
> > > > > I hadn't been following the discussions regarding 0.8 and
> replication
> > > > for a
> > > > > little while and this was a great post to refresh my memory and get
> > up
> > > to
> > > > > speed on the current replication architecture's design.
> > > > >
> > > > > --
> > > > > Felix
> > > > >
> > > > >
> > > > > On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao <junrao@gmail.com>
wrote:
> > > > >
> > > > > > I just posted the following blog on Kafka replication. This
may
> > > answer
> > > > > some
> > > > > > of the questions that a few people have asked in the mailing
list
> > > > before.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message