incubator-kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: producer queue size
Date Tue, 27 Nov 2012 01:02:43 GMT
I wonder if this makes sense for the default. I agree that most people who
are choosing async would prefer to drop the message rather than block since
it is async. But I wonder if we should default to -1 anyway and let them
set this. The reason is that I think benchmarking and other cases always
run into this and it matches the behavior of other messaging systems.

-Jay


On Mon, Nov 26, 2012 at 4:40 PM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> This will happen if you push data to the producer at a higher rate
> than it is able to send to the server.
> queue.size allows you to configure the size of the producer queue
> (defaults to 10K). Also,
> queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour
> instead of the producer throwing
> QueueFullExceptions.
>
> Thanks,
> Neha
>
> On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <jjkoshy.w@gmail.com> wrote:
> > To use async, set producer.type to async ; The default queue size is
> 10000;
> > and the default batch size is 200.
> >
> >
> > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <jamie.wang@actuate.com>
> wrote:
> >
> >> Hi, I am running the console demo comes with 0.7.2  in getting started
> >> guide. All is working fine. Then I use stdio redirect a file of 30K
> lines
> >> of messages into the producer and I received  an error "ERROR Event
> queue
> >> is full of unsent messages, could not send event:..." and exception
> stack
> >> shows:
> >> Exception in thread "main" kafka.producer.async.QueueFullException:
> Event
> >> queue is full of unsent messages, could not send event:...
> >>
> >> Is there a way in the produce.properties to configure the size of
> producer
> >> queue?   I know the producer is running in synchronous mode. How do I
> solve
> >> this problem or my usage is totally wrong?
> >>
> >> Thanks
> >> Jamie
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message