Hi Karl,
Outside - Karl's ACM wrote:
>> From: trustin@gmail.com [mailto:trustin@gmail.com]
>>
>> So.. the introduction of CompositeByteArray makes direct exposure of NIO
>> ByteBuffers somewhat difficult, although workarounds (disconnected
>> buffer by on-demand/manual copy or returns an array of buffers) are
>> available.
>>
>> WDYT?
>
> IMO the most essential use of multiple buffers is in the Scatter/Gather interfaces. So
at least an array returning method is necessary.
>
> I'm also quite interested in the zero-copy front. Grizzly and MINA currently differ in
this regard. I think that Grizzly without some modification has no support of zero-copy making
it unsuitable for high volume data, for example something like a pure Java logging server
with some kind of analysis/load to consume the data. I can begin testing this use case in
MINA, but I have some ideas to make the memory utilization more manageable.
>
> In this use case some slices of buffers could persist for longer than others. One approach
to minimize copies and allow more memory to be reclaimed would include evaluating allocating
smaller buffers (perhaps 512 bytes or smaller, varied as part of the experiment) and persisting
these until the data has been evaluated. These buffers could be periodically tested for their
waste and copied to free up the waste. However, to keep reads efficient I would want to experiment
with the GatheringByteChannel methods.
>
> For Grizzly I think this means creating a new ProtocolFilter and extending some part
of the entire filter chain to support it. For MINA I will have to do a little more reading.
Grizzly will also be relying entirely on the garbage collector to reclaim buffers and I may
have to craft a class to track buffer splits, and by then I might as well implement a buffer
pool.
Heap buffer pooling has been always slower than relying on garbage
collector from my experience. Direct buffer is somewhat different so it
requires more experimentation.
Thanks,
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
|