kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: ProducerPool - KAFKA-240
Date Tue, 31 Jan 2012 15:48:13 GMT
Joe,

For kafka-240, let's just focus on replacing SyncProducer with the new wire
protocol. There is a separate jira kafka-253 on producer-side refactoring.
We can defer other changes that you want to make there.

Thanks,

Jun

On Mon, Jan 30, 2012 at 9:36 PM, Joe Stein <cryptcom@gmail.com> wrote:

> I am not back from my length vacation until tomorrow (really Wednesday but
> going to get back to things tomorrow night) but anyways I just started to
> look at where I left off with the Producer Request changes....
>
> Looking at the ProducerPool I am not sure what I am thinking of doing with
> the brokerID is what others were thinking should be done or could be done
> differently.
>
> At first glance it seems best to make the ProducerPool from this
>
>  class ProducerPoolData[V](topic: String,
>                            bidPid: Partition,
>                            data: Seq[V]) {
>
> into this
>
>  class ProducerPoolData(brokerID: Int,producerRequest: ProducerRequest)
>
> and refactoring accordingly?
>
> not sure what is lost by doing the message serialization prior [V] moving
> up in the layers to this but have to get back digging in the code how that
> would look
>
> then
>
> class ProducerData[K, V](private val topic: String,
>                         private val key: K,
>                         private val data: java.util.List[V]) {
>
> would change to
>
> class ProducerData[K](
>                         private val key: K,
>                         private val producerRequest: ProducerRequest) {
>
> etc...
>
> at first blush it almost seems like ProducerPoolData can go away and some
> additions to producerRequest to handle things... but I am tired and need
> some rest from my vacation :)
>
> let me know, will jump back into this tomorrow night and then back again on
> Thursday/Friday hopefully (including tests) can get the ProducerRequest
> wrapped up by this coming Monday (if not sooner we'll see what other if any
> hurdles come about)
>
> Thanks!
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message