qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marnie McCormack <marnie.mccorm...@googlemail.com>
Subject Re: Qpid CLIENT_ACKNOWLEDGE
Date Thu, 24 Mar 2011 10:42:44 GMT
Hi Rajith,

Thanks (and oops).

On the stats for sync-publish vs transactions, guessing these are all
transient -> with the Java broker ?

For persistent messages, which you'd need for reliable delivery with the
Java broker, the figures would be closer together since the I/O write is the
larger cost.

Thanks,
Marnie

On Wed, Mar 23, 2011 at 8:20 PM, Rajith Attapattu <rajith77@gmail.com>wrote:

> Marnie,
>
> Looks like you forgot to hit replyAll.
> So I am forwarding your email to the list along with my comments.
>
>
>  On Wed, Mar 23, 2011 at 12:47 PM, Marnie McCormack <
> marnie.mccormack@googlemail.com> wrote:
>
>> Sorry, which broker are you usiong Sergey ? I'd assumed the Java broker
>> ...
>>
>
> Since the Java broker supports 0-10 now, you could use sync_publish.
>
>
>>
>> I'd probably disagree that you can achieve guaranteed delivery without a
>> transaction or its equivalent (which sync publish effectively is ?).
>>
>
> Synchronous publishing is not equivalent to transactions, but I understand
> what you meant here. You were probably alluding to the fact that in both
> cases messages are published in a synchronous way. i.e When the tx completes
> (or when the send method returns in sync-publish) you know that the broker
> has acked the message.
>
> However transactions have a lot more overhead than a simple sync publish,
> hence a higher performance penalty with transactions.
> For example look at the following relative numbers for a producer/consumer
> pair using the same setup.
>
> sync publish       15k msg/sec
> tx size =2          48 msg/sec (thats right just 48 messages per sec)
> tx size=1000    14k msg/sec
>
> Also from an application point of view, non transactional code is more
> simple and easy to write than transactional code.
> I would normally stay away from transactions unless I really have to. It
> basically negates a lot of benefits of using messaging.
>
> Regards,
>
> Rajith
>
>
>>
>> Regards,
>> Marnie
>>
>>   On Wed, Mar 23, 2011 at 4:21 PM, Rajith Attapattu <rajith77@gmail.com>wrote:
>>
>>> Hi Sergey,
>>>
>>> For Qpid to guarantee 'at-least-once' delivery on publish, you don't need
>>> to use transactions.
>>>
>>> Broker failures
>>> -----------------
>>> (1) In a stand-alone-broker you need to use persistence to ensure that
>>> messages survive a broker crash.
>>> (2) In a clustered broker you may be able to get away without using
>>> persistence (*)
>>>
>>> (*) Only the c++ broker supports clustering.
>>>       If you don't use persistence, then in the event of a total cluster
>>> failure you will loose messages.
>>>       Therefore it depends on how much you can tolerate these occasional
>>> failures.
>>>
>>> Application/Client lib failures
>>> ---------------------------------
>>> In the event of a failure in the application that sends the messages, you
>>> may loose messages.
>>> i.e if there are unacked messages in the client libs replay buffer, they
>>> will be lost.
>>> You could avoid that by using
>>>
>>> For JMS client
>>> -------------------
>>> (1) Using synchronous publishing. Use
>>> -Dsync_publish={persistent|transient|all}.
>>> (2) Transaction - as Marnie pointed out.
>>>
>>> Performance wise there isn't much of a difference between option 1 & 2 if
>>> you use large batches (ex 1000 msg/batch) for transactions.
>>> But ff you use small tx batches then there is an appreciable difference
>>> in performance.
>>>
>>> Therefore I recommend you use -Dsync_ack=persistent.
>>> Also the application code is much simpler when writing non transactional
>>> code.
>>>
>>> For other clients
>>> --------------------
>>> If you can handle reliable replay at the application level then you could
>>> just use asynchronous publishing without worrying about application failure.
>>> Ex. If your application is retrieving orders from a queue and processing
>>> it before sending a reply, then you could ack the order messsage after the
>>> broker acks your reply message.
>>>
>>> Hope this helps.
>>>
>>> Rajith
>>>
>>>
>>> On Wed, Mar 23, 2011 at 11:21 AM, Marnie McCormack <
>>> marnie.mccormack@googlemail.com> wrote:
>>>
>>>> Hi Sergey,
>>>>
>>>> To achieve guaranteed message delivery you need to be using transactions
>>>> on
>>>> *publish*. Qpid then guarantees 'at least once' delivery. You'd offset
>>>> the
>>>> reliable delivery cost against the cost or message loss during a failure
>>>> from sending non-transacted messages. The use of transactions and
>>>> persistence for reliable messaging adds cost in the shape of I/O.
>>>>
>>>> However, you could opt not to use transactions on consume since you
>>>> cannot
>>>> conceptually lose a message during consumption - if there's a failure
>>>> and a
>>>> message ack doesn't get back to the broker then you'll simply get the
>>>> message back again on restart. This assumes that you listen/catch/handle
>>>> errors and exception on consume such that you don't have application
>>>> level
>>>> side effects.
>>>>
>>>> Regards,
>>>> Marnie
>>>>
>>>> On Tue, Mar 22, 2011 at 1:03 PM, <Sergey_Zhemzhitsky@troika.ru> wrote:
>>>>
>>>> > Hi there
>>>> >
>>>> > I'm working with qpid 0.8 by means of JMS API and I'm using
>>>> > CLIENT_ACKNOWLEDGE mode to get higher performance comparing to
>>>> > transactinal processing.
>>>> > As a rule performance is rather pure in both cases. In transactional
>>>> mode
>>>> > I can receive about 1.6K messages per second, in CLIENT_ACKNOWLEDGE
>>>> mode
>>>> > I'm able to receive about 3.6K messages per second and in
>>>> AUTO_ACKNOWLEDGE
>>>> > mode I'm able to receive about 70K messages per second.
>>>> >
>>>> > Are there any recommendations how to improve performance and get a
>>>> > guaranted message delivery?
>>>> >
>>>> >
>>>> > Best Regards,
>>>> > Sergey
>>>> > _______________________________________________________
>>>> >
>>>> > The information contained in this message may be privileged and conf
>>>> > idential and protected from disclosure. If you are not the original
>>>> intended
>>>> > recipient, you are hereby notified that any review, retransmission,
>>>> > dissemination, or other use of, or taking of any action in reliance
>>>> upon,
>>>> > this information is prohibited. If you have received this
>>>> communication in
>>>> > error, please notify the sender immediately by replying to this
>>>> message and
>>>> > delete it from your computer. Thank you for your cooperation. Troika
>>>> Dialog,
>>>> > Russia.
>>>> > If you need assistance please contact our Contact Center  (+7495) 258
>>>> 0500
>>>> > or go to www.troika.ru/eng/Contacts/system.wbp
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message