activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: KahaDB: message persistency fsync issue.
Date Mon, 14 Jan 2013 21:53:35 GMT
by default, it will do a sync before every reply to a send, but it tries to
batch up writes for concurrent producers.  The sync reply to a a producer
send blocks pending a sync write batch.

For a sanity just use <kahaDB directory="${activemq.data}/**kahadb" /> and
post your producer.
So you should expect calls to fsync (or whatever fd.sync[1] delegates to on
solaris, fsync I think) more frequently than the default checkpoint
interval (5s).

The line of code on trunk is:
https://github.com/apache/activemq/blob/trunk/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/DataFileAppender.java#L367


[1] java.io.FileDescriptor#sync

I just ran a sanity on trunk with org.apache.activemq.perf.KahaDBQueueTest
and it is doing the right thing.


On 14 January 2013 18:34, Karel Gardas <karel.gardas@centrum.cz> wrote:

>
> Hello,
>
> I'm experimenting with ActiveMQ persistency layer. I'm using recommended
> KahaDB. My part of configuration file looks:
>
>         <persistenceAdapter>
>             <kahaDB directory="${activemq.data}/**kahadb"
>                     journalMaxFileLength="32mb"
>                     enableJournalDiskSyncs="true"
>                     concurrentStoreAndDispatchTopi**cs="false"
>                     concurrentStoreAndDispatchQueu**es="false"
>             />
>         </persistenceAdapter>
>
>
> now, with this I would expect activemq to fsync kahadb's files on every
> delivered message from the producer to the broker. Am I right? If so, then
> I'm afraid this is not happening as my drive barely lights and yet producer
> is pushing a lot of messages to the broker. As I've been suspecting
> something wrong happens here and as I'm using Solaris on this experiment
> I've attached dtrace to the broker java process and run following script:
>
> dtrace -n 'pid$target:libc:fsync:entry { trace(fds[arg0].fi_pathname); }'
> -p 28406
>
> and indeed, I see dtrace reporting fsync usage on following files every 5
> seconds:
>
>   1  75475                      fsync:entry /export/home/karel/Downloads/*
> *CRISIS/ActiveMQ/apache-**activemq-5.7.0/data/kahadb/db.**redo
>   8  75475                      fsync:entry /export/home/karel/Downloads/*
> *CRISIS/ActiveMQ/apache-**activemq-5.7.0/data/kahadb/db.**data
>   9  75475                      fsync:entry /export/home/karel/Downloads/*
> *CRISIS/ActiveMQ/apache-**activemq-5.7.0/data/kahadb/db-**5.log
>
> so fsync is called every 5 seconds instead of what I would expect every
> delivered message. Please note I'm using persistent delivery mode on my
> producer of course.
>
> My question is: am I right assuming with this configuration above activemq
> should fsync every delivered message? If not, is there any other option how
> to force this behavior?
>
> Thanks a lot!
> Karel
>
>


-- 
http://redhat.com
http://blog.garytully.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message