From users-return-7808-apmail-qpid-users-archive=qpid.apache.org@qpid.apache.org Mon Mar 4 19:31:25 2013 Return-Path: X-Original-To: apmail-qpid-users-archive@www.apache.org Delivered-To: apmail-qpid-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 97235D847 for ; Mon, 4 Mar 2013 19:31:25 +0000 (UTC) Received: (qmail 57561 invoked by uid 500); 4 Mar 2013 19:31:25 -0000 Delivered-To: apmail-qpid-users-archive@qpid.apache.org Received: (qmail 57538 invoked by uid 500); 4 Mar 2013 19:31:25 -0000 Mailing-List: contact users-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@qpid.apache.org Delivered-To: mailing list users@qpid.apache.org Received: (qmail 57529 invoked by uid 99); 4 Mar 2013 19:31:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Mar 2013 19:31:25 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of richard.peter@raytheon.com designates 199.46.199.208 as permitted sender) Received: from [199.46.199.208] (HELO dfw-mailout2.raytheon.com) (199.46.199.208) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Mar 2013 19:31:19 +0000 Received: from tx-mailout1.directory.ray.com (tx-mailout1.directory.ray.com [147.25.138.100]) by dfw-mailout2.raytheon.com (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id r24JUvkA030667 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Mon, 4 Mar 2013 19:30:57 GMT Received: from mk2-msg02.raymail.ray.com (mk2-msg02.mck.us.ray.com [138.126.190.142]) by tx-mailout1.directory.ray.com (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id r24JUu3r025865 sender richard.peter@raytheon.com for ; Mon, 4 Mar 2013 19:30:57 GMT Received: from dev04.oma.us.ray.com ([147.18.136.17]) by mk2-msg02.raymail.ray.com (Lotus Domino Release 8.5.3FP1) with ESMTP id 2013030413305662-650 ; Mon, 4 Mar 2013 13:30:56 -0600 Message-ID: <5134F670.6050408@raytheon.com> Date: Mon, 04 Mar 2013 13:30:56 -0600 From: Richard Peter User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: users@qpid.apache.org Subject: Re: 0.18 Java broker not persisting message to derby store References: <5134BD6B.3040701@raytheon.com> <5134DFEC.9010309@raytheon.com> In-Reply-To: X-MIMETrack: Itemize by SMTP Server on MK2-MSG02/SRV/Raytheon(Release 8.5.3FP1|March 07, 2012) at 03/04/2013 13:30:56, Serialize by Router on MK2-MSG02/SRV/Raytheon(Release 8.5.3FP1|March 07, 2012) at 03/04/2013 13:30:57, Serialize complete at 03/04/2013 13:30:57 X-TNEFEvaluated: 1 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-Spam-Score: 0.00% X-Virus-Checked: Checked by ClamAV on apache.org I should say it didn't as of version 0.8. We are upgrading from Qpid C++ Broker 0.8 with java/c++/python 0.8 clients, which did not require persistent to be set on the message. Our project has been running this way for a couple years and persistent recovery has worked perfectly. Richard On 03/04/2013 12:09 PM, Rajith Attapattu wrote: > On Mon, Mar 4, 2013 at 12:54 PM, Richard Peter > wrote: >> Ok, this wasn't an issue with the C++ broker, if a queue was marked durable >> any message was persisted. Thanks for the clarification. > This is not correct. You need to mark the the message as persistent in > order for it to be written to disk. > > Rajith > >> >> On 03/04/2013 10:07 AM, Robbie Gemmell wrote: >>> Hi Richard, >>> >>> The queue statistics suggest that you are not sending persistent messages: >>> "persistentEnqueuedMessages" : 0, >>> "totalEnqueuedMessages" : 161134, >>> >>> Non-persistent messages are only ever stored in memory on the broker, and >>> it will only recover persistent messages which are sent to durable queues. >>> >>> Robbie >>> >>> >>> On 4 March 2013 15:27, Richard Peter wrote: >>> >>>> I noticed that 0.18 Java Broker does not appear to be persisting message >>>> to the derby store. Running tests to verify memory thresholds in a >>>> disaster recovery situation. Had 100k+ messages on a durable queue with >>>> no >>>> consumers. When I bounced QPID, the memory usage was almost none and >>>> noticed the log wasn't recovering any messages: >>>> >>>> QPID2013-03-04 09:19:03,789 INFO [main] (Log4jMessageLogger.java:73) - >>>> [Broker] [vh(/edex)/ms(**DerbyMessageStore)] TXN-1004 : Recovery Start >>>> 2013-03-04 09:19:03,808 INFO [main] (Log4jMessageLogger.java:73) - >>>> [Broker] [vh(/edex)/ms(**DerbyMessageStore)] TXN-1005 : Recovered 0 >>>> messages for queue external.dropbox >>>> 2013-03-04 09:19:03,809 INFO [main] (Log4jMessageLogger.java:73) - >>>> [Broker] [vh(/edex)/ms(**DerbyMessageStore)] TXN-1006 : Recovery Complete >>>> : external.dropbox >>>> 2013-03-04 09:19:03,817 INFO [main] (Log4jMessageLogger.java:73) - >>>> [Broker] [vh(/edex)/ms(**DerbyMessageStore)] TXN-1006 : Recovery Complete >>>> >>>> >>>> Dug a little more and none of my durable queues are persisting. Anything >>>> I'm missing? I do have the store configured in my virtualhosts.xml: >>>> >>>> >>>> org.apache.qpid.server.**store.derby.DerbyMessageStore<**/class> >>>> ${QPID_WORK}**/messageStore >>>> >>>> >>>> Here is a sample from the rest interface: >>>> >>>> { >>>> "id" : "505024de-642d-3572-a80c-**750a1b9819e9", >>>> "name" : "external.dropbox", >>>> "state" : "ACTIVE", >>>> "durable" : true, >>>> "lifetimePolicy" : "PERMANENT", >>>> "type" : "standard", >>>> "exclusive" : false, >>>> "maximumDeliveryAttempts" : 0, >>>> "queueFlowControlSizeBytes" : 0, >>>> "queueFlowResumeSizeBytes" : 0, >>>> "queueFlowStopped" : false, >>>> "alertThresholdMessageAge" : 60000, >>>> "alertThresholdMessageSize" : 2117632, >>>> "**alertThresholdQueueDepthBytes" : 5294080, >>>> "**alertThresholdQueueDepthMessag**es" : 10000, >>>> "alertRepeatGap" : 30000, >>>> "statistics" : { >>>> "unacknowledgedBytes" : 0, >>>> "bindingCount" : 2, >>>> "queueDepthBytes" : 12697432, >>>> "queueDepthMessages" : 161134, >>>> "totalEnqueuedBytes" : 12697432, >>>> "persistentDequeuedBytes" : 0, >>>> "consumerCount" : 0, >>>> "unacknowledgedMessages" : 0, >>>> "consumerCountWithCredit" : 0, >>>> "totalDequeuedBytes" : 0, >>>> "totalDequeuedMessages" : 0, >>>> "persistentEnqueuedMessages" : 0, >>>> "totalEnqueuedMessages" : 161134, >>>> "persistentEnqueuedBytes" : 0, >>>> "persistentDequeuedMessages" : 0 >>>> }, >>>> "bindings" : [ { >>>> "id" : "d24e4587-ee99-3086-bc34-**056be3226f09", >>>> "name" : "external.dropbox", >>>> "durable" : true, >>>> "lifetimePolicy" : "PERMANENT", >>>> "exchange" : "amq.direct", >>>> "queue" : "external.dropbox", >>>> "arguments" : { >>>> } >>>> }, { >>>> "id" : "ac57bb00-a0aa-3c4e-a6d9-**276d3d520f39", >>>> "name" : "external.dropbox", >>>> "durable" : true, >>>> "lifetimePolicy" : "PERMANENT", >>>> "exchange" : "<>", >>>> "queue" : "external.dropbox", >>>> "arguments" : { >>>> } >>>> } ] >>>> } >>>> >>>> Thanks, >>>> Richard >>>> >>>> Thanks, >>>> Richard >>>> >>>> ------------------------------**------------------------------**--------- >>>> To unsubscribe, e-mail: >>>> users-unsubscribe@qpid.apache.**org >>>> For additional commands, e-mail: users-help@qpid.apache.org >>>> >>>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org >> >> For additional commands, e-mail: users-help@qpid.apache.org >> > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org > For additional commands, e-mail: users-help@qpid.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org