kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <crypt...@gmail.com>
Subject ProducerPool - KAFKA-240
Date Tue, 31 Jan 2012 05:36:43 GMT
I am not back from my length vacation until tomorrow (really Wednesday but
going to get back to things tomorrow night) but anyways I just started to
look at where I left off with the Producer Request changes....

Looking at the ProducerPool I am not sure what I am thinking of doing with
the brokerID is what others were thinking should be done or could be done

At first glance it seems best to make the ProducerPool from this

 class ProducerPoolData[V](topic: String,
                            bidPid: Partition,
                            data: Seq[V]) {

into this

 class ProducerPoolData(brokerID: Int,producerRequest: ProducerRequest)

and refactoring accordingly?

not sure what is lost by doing the message serialization prior [V] moving
up in the layers to this but have to get back digging in the code how that
would look


class ProducerData[K, V](private val topic: String,
                         private val key: K,
                         private val data: java.util.List[V]) {

would change to

class ProducerData[K](
                         private val key: K,
                         private val producerRequest: ProducerRequest) {


at first blush it almost seems like ProducerPoolData can go away and some
additions to producerRequest to handle things... but I am tired and need
some rest from my vacation :)

let me know, will jump back into this tomorrow night and then back again on
Thursday/Friday hopefully (including tests) can get the ProducerRequest
wrapped up by this coming Monday (if not sooner we'll see what other if any
hurdles come about)


Joe Stein
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message