logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Sicker <boa...@gmail.com>
Subject Experimenting with NIO
Date Sat, 25 Feb 2017 23:19:04 GMT
We already use a bit of NIO (ByteBuffer for layouts and appenders/managers,
MappedByteBuffer for mmap'd files, FileLock for locking files, etc.), and
I've been playing around with the NIO API lately. I have some sample code
here <https://github.com/jvz/nio-logger> to show some trivial use case of
AsynchronousFileChannel. In Java 7, there is also AsynchronousSocketChannel
which could theoretically be used instead of adding Netty for a faster
socket appender. In that regard, I'm curious as to how useful it would be
to have similar appenders as the OutputStream ones, but instead using
WritableByteChannel, GatheringByteChannel (possible parallelization of file
writing?), and the async channels (there's an AsynchronousByteChannel
class, but I think they screwed this one up as only one of the three async
channel classes implements it).

Another related issue I've seen is that in a message-oriented appender
(e.g., the Kafka one), being able to stream directly to a ByteBuffer is not
the right way to go about encoding log messages into the appender. Instead,
I was thinking that a pool of reusable ByteBuffers could be used here where
a ByteBuffer is borrowed on write and returned on completion (via a
CompletionHandler callback). The Kafka client uses a similar strategy for
producing messages by dynamically allocating a pool of ByteBuffers based on
available memory.

Also, I don't have much experience with this, but if we had a pool of
reusable ByteBuffers, could we use direct allocation to get off-heap
buffers? That seems like an interesting use case.

-- 
Matt Sicker <boards@gmail.com>

Mime
View raw message