From users-return-8736-apmail-qpid-users-archive=qpid.apache.org@qpid.apache.org Mon Aug 5 20:53:34 2013 Return-Path: X-Original-To: apmail-qpid-users-archive@www.apache.org Delivered-To: apmail-qpid-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 002B8C314 for ; Mon, 5 Aug 2013 20:53:33 +0000 (UTC) Received: (qmail 69682 invoked by uid 500); 5 Aug 2013 20:53:33 -0000 Delivered-To: apmail-qpid-users-archive@qpid.apache.org Received: (qmail 69628 invoked by uid 500); 5 Aug 2013 20:53:33 -0000 Mailing-List: contact users-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@qpid.apache.org Delivered-To: mailing list users@qpid.apache.org Received: (qmail 69620 invoked by uid 99); 5 Aug 2013 20:53:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Aug 2013 20:53:33 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rob.j.godfrey@gmail.com designates 209.85.214.170 as permitted sender) Received: from [209.85.214.170] (HELO mail-ob0-f170.google.com) (209.85.214.170) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Aug 2013 20:53:27 +0000 Received: by mail-ob0-f170.google.com with SMTP id eh20so6717699obb.15 for ; Mon, 05 Aug 2013 13:53:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=PHWOBTsrD3LRTwt0qvbs7HOn2tq+KdTUe6HXEWeIq88=; b=QNZFrRpfzpjwVPs5dOcD6LGI8lEn+e6crQJHJ78ICEQEU5aNndc/SgCgGoQRfEEPY2 0Cqk6pNABB6hTY1R9/LDXsJvMNjtSJsxdQOaqVApjohu4tmXUJtHtkZ3sUITTK6Hvw/M G4GAm7b+KHrFIAkVzxhoGl5qJRW/jRPC2jNKXWMLv8d6GAEZJAnnPDMScCcDkXQvm0a2 6RZdK7jNiloHlfetQX3btgKvkta5uRU3wgz1ep64fGUSg03x93vC4yTlo/7xlie5/dFU MMBn5VMALUiNZw8C9D5qUhh5TmvBpPy4n7PzYPTkHeDeGkPfihcnRzxCyXicG0iQuNI3 n22g== MIME-Version: 1.0 X-Received: by 10.182.28.98 with SMTP id a2mr15546087obh.36.1375735985898; Mon, 05 Aug 2013 13:53:05 -0700 (PDT) Received: by 10.60.165.70 with HTTP; Mon, 5 Aug 2013 13:53:05 -0700 (PDT) In-Reply-To: <1375731610189-7596486.post@n2.nabble.com> References: <1375718156499-7596467.post@n2.nabble.com> <1375731610189-7596486.post@n2.nabble.com> Date: Mon, 5 Aug 2013 22:53:05 +0200 Message-ID: Subject: Re: Automatic queue cleanup From: Rob Godfrey To: "users@qpid.apache.org" Content-Type: multipart/alternative; boundary=089e0158adfebe8e3604e3397eb2 X-Virus-Checked: Checked by ClamAV on apache.org --089e0158adfebe8e3604e3397eb2 Content-Type: text/plain; charset=ISO-8859-1 Hi James, if you are using persistent messages sent to durable queues then the Java Broker should be able to evict the message content from memory when low memory conditions are reached (though it does retain some overhead for meta data relating to the entry of the message on the queue). Messages not marked as persistent will retain all their content in memory (and thus if not restricted by other means, such as TTL, or producer flow control), excessive queue size may cause out of memory exceptions. In terms of controlling the amount of message data retained by the broker in memory, there are currently a number of steps you could take... firstly - as above - ensuring that you are using persistent messages. Secondly you can use TTL to cause message expiry (if this is appropriate for your use case). You can also use producer flow control to enforce (approximate) limits on how large queues can grow. Unfortunately the Java Broker does not yet implement all the queue policies that the C++ Broker implements... in particular there is no way of limiting the size or depth of a queue, and discarding messages in either a FIFO or LIFO manner - is this the sort of functionality you would be looking for? I know Alex produced a patch at some point for implementing ring queues in the Java Broker, but we've not yet seen a great deal of demand for this behaviour... however I believe Fraser outlined a use case where it might be useful, and possibly you also have such a use case -- Rob On 5 August 2013 21:40, jbelch wrote: > Keith, > > We have increased it to 2GB in the qpid-server. We could continue to > increase it, but we wanted to figure out if there was a way to get the Qpid > broker to cleanup. We are using Berkeley DB as the message store and we > only have two queues marked as durable right now. The errors I see right > before it crashes are: > > MESSAGE_COUNT_ALERT on Queue - 180: Maximum count on queue > threshold > (50) breached > MESSAGE_AGE_ALERT on Queue - 343925s: Maximum age on queue > threshold > (3600s) breached > > > > -- > View this message in context: > http://qpid.2158936.n2.nabble.com/Automatic-queue-cleanup-tp7596467p7596486.html > Sent from the Apache Qpid users mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org > For additional commands, e-mail: users-help@qpid.apache.org > > --089e0158adfebe8e3604e3397eb2--