qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen M <lefthandma...@gmail.com>
Subject override to turn BDBMessageStore syncCommit to false?
Date Sat, 14 Jan 2012 02:27:26 GMT
Hi,

    I would like to exploit the Berkeley DB's nosyncCommit (lazy commit)
option to exploit enqueue performance.
I see that in the current BDBMessageStore implementation there is no way to
override the syncCommit Boolean value and
would like to know if this could be made into an option.

Also, that said It'll be nice to know if someone had used the
nosyncCommit() option that BDB provides and if using it does any invisible
harm. (DB corruptions etc.).

Now, to why do I want to override that value to false? (I do realize that
turning it to false, doesn't guarantee that the message was written to disk)
In my use case. I have a database outside Qpid which persists all the
messages enqueued (for durability and transactionality with message
callback operations.).

My sequence of operations is,

1) Enqueue messages -> persist message in my database outside Qpid ->
enqueue to Qpid in persistant mode
Since my message is already persisted in my database, I'm ok with losing a
message in BDB, and would like to enhance throughput of my overall enqueue
operation by using the nosyncCommit option.
The reason I'm not using non-persistant mode in Qpid is because I don't
want to overwhelm the memory available by flooding the broker with messages.

In the event of a broker crash, I plan to restore the messages from my
database outside Qpid. (I realize this could turn out a little expensive).
I've been thinking
that if there is a replication/HA solution to have messages go to a primary
and secondary broker (in a 2 instance cluster), in the event of a crash,
I'll have to
only restore the most recent messages from my database outside Qpid upto a
certain checkpoint to which the primary and secondary are sync'ed (Not sure
if this is achievable.).

I do realize that my architecture is a little complicated. Please do let me
know if you have more questions about what I'm trying to achieve here, I'll
be happy to
clarify doubts.

I'd really like to know if I can add an option to override the syncCommit
to false without any major side-effects apart from what I expect.


Thank you,
-- 
-Praveen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message