arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corey J Nolet <>
Subject Understanding "shared" memory implications
Date Thu, 10 Mar 2016 05:25:05 GMT
If I understand correctly, Arrow is using Netty underneath which is using Sun's Unsafe API
in order to allocate direct byte buffers off heap. It is using Netty to communicate between
"client" and "server", information about memory addresses for data that is being requested.

I've never attempted to use the Unsafe API to access off heap memory that has been allocated
in one JVM from another JVM but I'm assuming this must be the case in order to claim that
the memory is being accessed directly without being copied, correct?

The implication here is huge. If the memory is being directly shared across processes by them
being allowed to directly reach into the direct byte buffers, that's true shared memory. Otherwise,
if there's copies going on, it's less appealing.


Sent from my iPad
View raw message