logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Sicker <boa...@gmail.com>
Subject Re: Experimenting with NIO
Date Sun, 26 Feb 2017 16:05:45 GMT
Perhaps something got optimized by the JVM? I'll add some JMH tests to this
repo to try out various approaches.

On Sat, Feb 25, 2017 at 21:12, Apache <ralph.goers@dslextreme.com> wrote:

> I tried using a FileChannel for the FileAppender a week or so ago to see
> if passing the ByteBuffer to the FileChannel would improve performance
> since it doesn’t have to be synchronized. I didn’t see any improvement
> though and I ended up reverting it. But I might have done something wrong.
>
> Ralph
>
> On Feb 25, 2017, at 4:19 PM, Matt Sicker <boards@gmail.com> wrote:
>
> We already use a bit of NIO (ByteBuffer for layouts and
> appenders/managers, MappedByteBuffer for mmap'd files, FileLock for locking
> files, etc.), and I've been playing around with the NIO API lately. I have
> some sample code here <https://github.com/jvz/nio-logger> to show some
> trivial use case of AsynchronousFileChannel. In Java 7, there is also
> AsynchronousSocketChannel which could theoretically be used instead of
> adding Netty for a faster socket appender. In that regard, I'm curious as
> to how useful it would be to have similar appenders as the OutputStream
> ones, but instead using WritableByteChannel, GatheringByteChannel (possible
> parallelization of file writing?), and the async channels (there's an
> AsynchronousByteChannel class, but I think they screwed this one up as only
> one of the three async channel classes implements it).
>
> Another related issue I've seen is that in a message-oriented appender
> (e.g., the Kafka one), being able to stream directly to a ByteBuffer is not
> the right way to go about encoding log messages into the appender. Instead,
> I was thinking that a pool of reusable ByteBuffers could be used here where
> a ByteBuffer is borrowed on write and returned on completion (via a
> CompletionHandler callback). The Kafka client uses a similar strategy for
> producing messages by dynamically allocating a pool of ByteBuffers based on
> available memory.
>
> Also, I don't have much experience with this, but if we had a pool of
> reusable ByteBuffers, could we use direct allocation to get off-heap
> buffers? That seems like an interesting use case.
>
> --
> Matt Sicker <boards@gmail.com>
>
>
> --
Matt Sicker <boards@gmail.com>

Mime
View raw message