From users-return-2227-apmail-qpid-users-archive=qpid.apache.org@qpid.apache.org Mon Nov 23 12:18:41 2009 Return-Path: Delivered-To: apmail-qpid-users-archive@www.apache.org Received: (qmail 24718 invoked from network); 23 Nov 2009 12:18:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 23 Nov 2009 12:18:40 -0000 Received: (qmail 57336 invoked by uid 500); 23 Nov 2009 12:18:40 -0000 Delivered-To: apmail-qpid-users-archive@qpid.apache.org Received: (qmail 57301 invoked by uid 500); 23 Nov 2009 12:18:40 -0000 Mailing-List: contact users-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@qpid.apache.org Delivered-To: mailing list users@qpid.apache.org Received: (qmail 57291 invoked by uid 99); 23 Nov 2009 12:18:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Nov 2009 12:18:40 +0000 X-ASF-Spam-Status: No, hits=-4.0 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of gsim@redhat.com designates 209.132.183.28 as permitted sender) Received: from [209.132.183.28] (HELO mx1.redhat.com) (209.132.183.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Nov 2009 12:18:31 +0000 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nANCI9mM026130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 23 Nov 2009 07:18:09 -0500 Received: from [10.11.11.147] (vpn-11-147.rdu.redhat.com [10.11.11.147]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nANCI8ie001511 for ; Mon, 23 Nov 2009 07:18:08 -0500 Message-ID: <4B0A7D11.5030807@redhat.com> Date: Mon, 23 Nov 2009 12:16:17 +0000 From: Gordon Sim Organization: Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom.,Registered in England and Wales under Company Registration No. 3798903,Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland) User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4pre) Gecko/20091014 Fedora/3.0-2.8.b4.fc11 Thunderbird/3.0b4 MIME-Version: 1.0 To: users@qpid.apache.org Subject: Re: Request to Add autoDelete Parameter to qpid-route command References: <951EAAA951E3AD4B82B5B30AA2A0AF2BE41D60962D@Commitchs1.commitent.com> In-Reply-To: <951EAAA951E3AD4B82B5B30AA2A0AF2BE41D60962D@Commitchs1.commitent.com> Content-Type: multipart/mixed; boundary="------------040504030606010200040205" X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Virus-Checked: Checked by ClamAV on apache.org --------------040504030606010200040205 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit On 11/20/2009 08:50 PM, Cullen Davis wrote: > Synopsis: > Dynamic federation does not guarantee delivery of messages. Setting autoDelete=false on the “transit-queue” during dynamic federation queue creation would eliminate the issue. > > Summary: > Our solution utilizes Qpid 0.5 C++ brokers and clients running on Red Hat Enterprise Linux 5.4. We are a government backed project that must deploy onto networks that are low bandwidth, high latency, and intermittently disconnected. Guaranteed delivery of the messages is a fundamental requirement. Our solution depends on brokers using direct exchanges and being dynamically federated. Using dynamic federation, when network connectivity is lost for more than seven minutes, all messages awaiting federation are deleted. At re-connect, the federation routes are rebuilt but the unsent messages are not restored. (Originally Stated in users mailing list on Thu, 05 Nov, 16:55) > > Change Request: > Allow Dynamic Federation to create “transit-queues” with autoDelete=False. We would like to request a new option, autoDelete, on the qpid-route command. In our case, we would run > qpid-route --durable --autodelete=false dynamic add brokerB brokerA fed.direct > ============ > This would be used to create exchange bindings with transit-queues having the autoDelete property set to false. I'd argue that if you are specifying durable that implies you don't want auto deletion (durable is really only of any value in recovering from loss of in-memory state and in that case you would have lost the session and triggered auto-delete anyway). (The attached trivial patch would do this I believe, though perhaps it unnecessarily couples the durability of the bridge meta-data with the durability of the queue used in which case a 'reliable' option might be prefered). Having a timeout for a queue such that if there is no subscriber for x seconds then the queue gets deleted might be a useful addition. That would allow some protection over the case where a failed link never comes back (or doesn't come back fast enough to avoid a critical build up of messages). > Details: > We federate our brokers as follows: > qpid-route --durable dynamic add brokerB brokerA fed.direct > > The dynamic option causes the qpid-route command to create a new “transit-queue”, with the name "bridge-queue" at brokerA. The bridge-queue serves as the means to move messages from brokerA to brokerB. The new queue had queue properties of durable=False, exclusive=True and autoDelete=True. > > If the network connection is broken for more than seven minutes, the session on brokerB subscribing to the "bridge-queue" on brokerA is considered to have ended. The loss of the session subscriber triggers the the autoDelete of the bridge-queue. The result is that all messages awaiting federation via the bridge queue are deleted (NOT persisted at all). > > Ted Ross suggested a work around. He suggested we create a “queue” route where the destination broker subscribes to an existing queue on the source broker. This should cause the inter-broker route to use message acknowledgement in such a way that recovery will be clean. > > We encountered the following issues with the queue route: > 1) “Out-of-order” message delivery was experienced > 2) The first 1000KB of message data (in our case 10 messages) were not delivered until the broker process was restarted. This sounds crazy but we replicated the case multiple times. That sounds like a session is not being correctly cleaned up perhaps and there are some unacked messages in a locked state. Would you be willing to raise a Jira for this and list the steps taken to reproduce? > 3) Our solution loses the ability to do dynamic binding. > > Our customer needs to have the in-order, guaranteed delivery of messages on low quality networks. > > How should we proceed with this request? > > > Cullen Davis > CommIT Enterprises, Inc > > > --------------------------------------------------------------------- > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:users-subscribe@qpid.apache.org > --------------040504030606010200040205 Content-Type: text/x-patch; name="durable_bridge.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="durable_bridge.patch" diff --git a/qpid/cpp/src/qpid/broker/Bridge.cpp b/qpid/cpp/src/qpid/broker/Bridge.cpp index 79e311d..9e7b3bb 100644 --- a/qpid/cpp/src/qpid/broker/Bridge.cpp +++ b/qpid/cpp/src/qpid/broker/Bridge.cpp @@ -131,7 +131,7 @@ void Bridge::create(Connection& c) queueSettings.setString("qpid.trace.exclude", localTag); } - bool durable = false;//should this be an arg, or would we use srcIsQueue for durable queues? + bool durable = args.i_durable; bool autoDelete = !durable;//auto delete transient queues? peer->getQueue().declare(queueName, "", false, durable, true, autoDelete, queueSettings); if (!args.i_dynamic) --------------040504030606010200040205 Content-Type: text/plain; charset=us-ascii --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscribe@qpid.apache.org --------------040504030606010200040205--