activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nigro_franz <>
Subject Re: sun.misc.Unsafe VS io.netty.util.internal.PlatformDependent VS null
Date Fri, 10 Jun 2016 07:31:43 GMT

Thanks for the good advice, i'm happy to let Unsafe improve further the
performances of this mapped implementation 
<>   there is the JIRA
related to the memory mapped implementation of SequenceFile: i need to
squash the (too many) commits into one before doing any PR and obviosly
reimplement a "confined" version of PlatformDependent that let Android too
to use the mmap implementation (crossing the fingers, ART VM is pure evil).
I've built for now only 2 benchmarks (latency focused), but without using
JHM because of the coordinated omission issue, thus i've found JLBH (Java
Latency Benchmark Harness)
that is best suited for this kind of measurements IMHO. 
The results of the benchmarks are pretty good against NIO (if you push data
in "burst" of less then 10% of your memory, for a default configured latest
ubuntu linux box): about 2 order of magnitude less in the 99 %tile latency
With these benchs i could measure how much on different configured HW the
mmap Sequence File could "keep the pace" with a sustained load of messages,
thus producing something pretty similar to a real use case eg without any
"math benefits" derived by the backpressure of the OS during the page
faulting unavoidable pauses.
I've planned to solve the Page Faulting issue using a pure async
implementation, using a single producer single consumer wait-free ringBuffer
that feeds an event loop responsible to write the data on the MappedFile but
the benchs that i've built (around Journal and SequenceFile) show that the
higher layers (above the mmap impl of SequenceFile) slow even this pure-sync
implementation, so i'm not sure that the Journal could receive a perceptible
benefit from an async one considering that it will only reduce the hiccups
when a page fault of few microseconds occour (anyway it could be tuned with
Huge Pages and increasing the dirty ratio).
I'm very curious to test it against libaio and i will be very happy if you
are interested to try it with me, in order to be objective: my expectations
are that with burst of small messages the mmap impl will be faster (crossing
JNI has higher cost than call an intrinsified tiny copy with Unsafe, in my
experience) , the opposite while the size of the messages grows. If you need
other benchs (throughput focused with JHM) or need more info about the
implementation i'm ever available 

View this message in context:
Sent from the ActiveMQ - Dev mailing list archive at

View raw message