activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nigro_franz <nigro....@gmail.com>
Subject Re: Paging
Date Mon, 27 Mar 2017 10:40:18 GMT
Sorry for the late response guys, I was killed by the spring allergies in the
last days; my eyes was burning :(


Clebert Suconic wrote
> I am treating Mappedfile as experimental still.. 

Agree, it is a real work in progress :) 
To use the mapped journal safely there is at least 1 big blind spot that
need to be addressed (with datasync on): major page faults.
Right now I've done some benchmarks around multi-threading append and the
biggest bottleneck seems to be way by which we cache the journal files: I'm
guessing will be necessary to leverage the OS page cache strategies and 
JournalFilesRepository
<https://github.com/apache/activemq-artemis/blob/master/artemis-journal/src/main/java/org/apache/activemq/artemis/core/journal/impl/JournalFilesRepository.java>
 
could manage the file caching with a LRU policy, but I need some help to
implement it..


Michael André Pearce wrote
> MemoryMapped for paging was actually the one I was more thinking to be
> beneficial. 

Yes, for Paging (or the normal Journal with datasync off), I'm sure that it
will be pretty fast as it is as long that there will be enough memory in the
HW.
If any of you is curious to try the latest version of it, here is my latest
branch on it:
https://github.com/franz1981/activemq-artemis/tree/batch_buffer.
On it you could find a  journal benchmark
<https://github.com/franz1981/activemq-artemis/blob/batch_buffer/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/JournalTptBenchmark.java>
 
that could show the page faults issue (on Nio and Mapped) and a general idea
of how it perform compared to the others (in multi thread or mixed producers
scenario).


Clebert Suconic wrote
> It will be awesome when people start usin new devices that are optimized 
> around mmap. 

Yes, it will be cool! As soon as possible I'll recontact the guys that
developed PMEM to try to make a working implementation ( it will be a
straightforward porting of the current MAPPED journal very likely), but i
need the hardware to try it for real :D







--
View this message in context: http://activemq.2283324.n4.nabble.com/Paging-tp4724085p4724130.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.

Mime
View raw message