Return-Path: X-Original-To: apmail-incubator-kafka-users-archive@minotaur.apache.org Delivered-To: apmail-incubator-kafka-users-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 897FBD6C7 for ; Tue, 27 Nov 2012 00:40:42 +0000 (UTC) Received: (qmail 41677 invoked by uid 500); 27 Nov 2012 00:40:42 -0000 Delivered-To: apmail-incubator-kafka-users-archive@incubator.apache.org Received: (qmail 41657 invoked by uid 500); 27 Nov 2012 00:40:42 -0000 Mailing-List: contact kafka-users-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kafka-users@incubator.apache.org Delivered-To: mailing list kafka-users@incubator.apache.org Received: (qmail 41647 invoked by uid 99); 27 Nov 2012 00:40:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 00:40:42 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of neha.narkhede@gmail.com designates 74.125.82.41 as permitted sender) Received: from [74.125.82.41] (HELO mail-wg0-f41.google.com) (74.125.82.41) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2012 00:40:37 +0000 Received: by mail-wg0-f41.google.com with SMTP id ds1so2149908wgb.0 for ; Mon, 26 Nov 2012 16:40:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=pvHQAWfipncDzK+O5Zh1+D8b0meFvyjL94zO61c7xAQ=; b=cn8qsBrbLFCVs/ROHy/IT951Q934urxX77B65A/Ud7ikQqnjiNUpOQGfKyTP9bqYnf y918s4K+p2mBBVqN8t6MVLkfd40EMA0AbS6ssYblHU9GTqvJlrQjFbU7Q8nzt1wbb8eB CqL6pCI1KzT20CbISit89Sz/ZVnzKu4kJ+1ZwnALAEMJLt92vhtA+LVEn2hNY+w6dwI4 gYxBkVWUF3l7BlAHqW0WuAX84EvTGI0AWEWtfk2xHNYUTrqbv0xtKSE6BvxCLwXKoaZA GYNBMA8sd22xiDd7TP8Ti5CWAty/2TiKxo8oRaDR2uPljCJCn8lgyYC7/WGq5Y9vpYck EH9g== MIME-Version: 1.0 Received: by 10.180.78.161 with SMTP id c1mr20574488wix.4.1353976815112; Mon, 26 Nov 2012 16:40:15 -0800 (PST) Received: by 10.216.142.198 with HTTP; Mon, 26 Nov 2012 16:40:14 -0800 (PST) In-Reply-To: References: Date: Mon, 26 Nov 2012 16:40:14 -0800 Message-ID: Subject: Re: producer queue size From: Neha Narkhede To: kafka-users@incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org This will happen if you push data to the producer at a higher rate than it is able to send to the server. queue.size allows you to configure the size of the producer queue (defaults to 10K). Also, queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour instead of the producer throwing QueueFullExceptions. Thanks, Neha On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy wrote: > To use async, set producer.type to async ; The default queue size is 10000; > and the default batch size is 200. > > > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang wrote: > >> Hi, I am running the console demo comes with 0.7.2 in getting started >> guide. All is working fine. Then I use stdio redirect a file of 30K lines >> of messages into the producer and I received an error "ERROR Event queue >> is full of unsent messages, could not send event:..." and exception stack >> shows: >> Exception in thread "main" kafka.producer.async.QueueFullException: Event >> queue is full of unsent messages, could not send event:... >> >> Is there a way in the produce.properties to configure the size of producer >> queue? I know the producer is running in synchronous mode. How do I solve >> this problem or my usage is totally wrong? >> >> Thanks >> Jamie >>