activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Green <>
Subject Re: Kahadb Corruption Scenarios
Date Fri, 10 Jun 2011 08:20:32 GMT

Don't forget there are hardware-level performance improvements from the
operating system down to the disks which may affect reliability.


On 8 June 2011 16:53, Ozan Seymen <> wrote:

> Hi all,
> Can you please explain the scenarios where we might have AMQ message
> storage (kahadb) corrupted in a way that it is not recoverable?
> In the solution I am working on, I simply cannot afford to lose any
> messages. In order to secure this, I will:
> *         Rely on transactions on the publisher side (both performance and
> reliability). This should ensure that I will catch exceptions for every
> batch. If commit succeeds, broker should assume responsibility and have
> persisted the message to disk (can you please confirm whether this is
> correct?)
> *         Implement durable queues/messages (deliverymode is persistent)
> *         Rely on acks on consumer side (dubs_ok probably).
> Even though all of these above prevent message loss in normal conditions,
> none of them covers the case where data gets corrupted in the broker. There
> is a window (albeit small) that things might go wrong: broker assumes
> responsibility (message is in the disk) but before message is sent to the
> consumer, broker experiences problems which corrupts the storage. If I can't
> bring the broker up with all data previously persisted, I've lost messages -
> producer forgot about the message as broker accepted responsibility and
> consumers have no idea about the messages since they haven't been delivered
> to it.
> So does kahadb have some sort of transaction log that it keeps which should
> recover messages? If not, would changing the message store to a JDBC
> provider (and use Postgre, MySql, SQL Server, etc) help?
> I would really appreciate any info you guys can share.
> Cheers,
> Ozan
> ________________________________
> This e-mail and any attached files are intended for the named addressee
> only. It contains information which may be confidential and legally
> privileged and also protected by copyright. Unless you are the named
> addressee (or authorised to receive for the addressee) you may not copy or
> use it or disclose it to anyone else. If you received it in error please
> notify the sender immediately and then delete it from your system.
> Please be advised that the views and opinions expressed in this e-mail may
> not reflect the views and opinions of The Digital Property Group Limited or
> any of its subsidiary companies.
> We make every effort to keep our network free from viruses. However, you do
> need to check this e-mail and any attachments to it for viruses as we can
> take no responsibility for any computer virus which may be transferred by
> way of this e-mail. We reserve the right to monitor all e-mail
> communications.
> The Digital Property Group Limited is a Daily Mail and General Trust plc
> company.
> Registered Office: Northcliffe House, 2 Derry Street, London, W8 5TT.
> Registered in England & Wales No: 02290527 VAT no. 243571174
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit
> ______________________________________________________________________

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message