incubator-kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nn chuong411 <>
Subject Re: Retrieve Partition-Id from MessageStream
Date Thu, 08 Dec 2011 05:54:41 GMT
Ok, Here is my usecase:
   -I use a group of producers to tail log for one MMO game, one producer
for one server game
   -Each producer will send messages to a particular partition under a
shared topic (mean, one topic for one game)
   -I use a static map table to map partition-id - server game name
   -A group of consumers will consume log for this game, one consumer for
at least one partition
   -Number of partitions for one consumer will be dynamic changed for load
balancing at consumer side
   -Processing at consumer require a know server name (mean, partition id)
for each message
   -I don't wanna handle offset updating manually with Simple - Consumer,
it risk my processing logic
So, do you have any suggestion for me?

On Thu, Dec 8, 2011 at 12:21 PM, Neha Narkhede <>wrote:

> No. Each message stream can have messages from multiple partitions and this
> list can change with every rebalancing operation. Returning this
> information to the user is a little tricky, but can you describe how you
> would like to use it ? This will require some thought, I don't know the
> right answer.
> Thanks,
> Neha
> On Wed, Dec 7, 2011 at 7:37 PM, nn chuong411 <>
> wrote:
> > Hi all,
> >    I'm using ZooKeeperConsumerConnector to create a group of
> > messageStream. But is there anyway to retrieve partitionId associated
> with
> > each messageStream?
> > --
> > Nguyen Nam Chuong
> > Software Engineer - VNG Cooperation
> > Ho Chi Minh City - Vietnam
> >

Nguyen Nam Chuong
Software Engineer - VNG Cooperation
Ho Chi Minh City - Vietnam

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message