activemq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (ARTEMIS-508) Sequential File Improvement + Performance Tests
Date Sat, 18 Jun 2016 03:52:05 GMT


ASF GitHub Bot commented on ARTEMIS-508:

GitHub user franz1981 opened a pull request:

    Mapped journal

    Memory Mapped Implementation of Sequential File + Benchs.
    Experimental GCFree Single Writer Journal + Benchs.
    Experimental Binary Encoding of Journal Header + Record using aligned sequential stride
to performe better on every arch.

You can merge this pull request into a Git repository by running:

    $ git pull mapped-journal

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #590
commit ed6ab0daf4be24fafbc77b95d319a5e5cab0eb13
Author: franz1981 <>
Date:   2016-06-08T13:21:12Z

    Merge pull request #1 from apache/master
    Update from original

commit 58f46ad701f7df27029f135aa994c0d9cf456fb1
Author: Francesco Nigro <>
Date:   2016-06-17T21:10:23Z

    Merge remote-tracking branch 'origin/master'

commit 62beb899877f7d252341f722de17a433db4e59a6
Author: Francesco Nigro <>
Date:   2016-06-18T03:47:41Z

    Added memory mapped impl of Sequential File + benchs.
    Added experimental GCFree Journal impl + benchs + Sequentially Encoded Aligned Binary


> Sequential File Improvement + Performance Tests
> -----------------------------------------------
>                 Key: ARTEMIS-508
>                 URL:
>             Project: ActiveMQ Artemis
>          Issue Type: Improvement
>            Reporter: Francesco Nigro
>              Labels: feature, performance, test
> I've implemented in the package a new memory
mapped SequentialFile implementation to support fast write/read operations for OSs that cannot
use libaio or need RAM access performances on standard Files.
> The implementation is not thread-safe (it is needed?) and needs more buffer checks (or
a complex implementation).  I'm using the Netty's PlatformDependent class to perform bulk
copy without safepoint's poll issues.
> The implementation is simple but it's good as a proof of concept to compare it against
the others: i' ve added a coordinated-omission performance test to measure the latency of
a directWrite + OS jitter.
> The write tests show performances typical of memory mapped file: quoting from Peter Lawrey
"for burst of up to 10% of the main memory, it can sustain rates of 1 - 3 GB/second written.
e.g. A laptop with 8 GB of memory might handle bursts of 800 MB at a rate of 1 GB per second.
A server with 64 GB of memory might handle a burst of 6.5 GB at a rate of 3 GB per second".
> I want to improve the quality of the implementation by:
> 1- enforcing the original SequenceFile contract 
> 2- replacing synchronized reads/writes with lock-free versions
> 3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue + EventLoop's
> 4- addressing all the False-Sharing issues around all the AtomicLong instances used
> 5- reducing the garbage produced for the fast paths to 0
> [6- expose try methods to allow direct flow control]
> [7- replacing the Semaphore based rate limiter with a lock free one]
> That's are all proposals, what do you think about it? 
> Regards

This message was sent by Atlassian JIRA

View raw message