pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] merlimat opened a new pull request #882: Use thread local to allocate temp byte[] instead of allocator
Date Thu, 01 Jan 1970 00:00:00 GMT
merlimat opened a new pull request #882: Use thread local to allocate temp byte[] instead of
allocator
URL: https://github.com/apache/incubator-pulsar/pull/882
 
 
   ### Motivation
   
   When serializing/deserializing protobuf, we use a temporary `byte[]` to extract data from
a direct byte buffer and wrap it in a protobuf `ByteString` object.
   
   The current implementation is using a `RecyclableHeapByteBuf` to get a heap buffer from
a pool and releasing it immediately in the same thread. 
   
   The problem is that the Netty ByteBuf allocator is allocating a new local heap arena for
each thread, using 16Mb `byte[]`. If there are many threads doing serialization/deserialization,
this can end up using a lot of heap mem: 16Mb per thread.
   
   Since we really need just 1 tiny temp buffer, we can achieve that more easily with a thread
local `byte[]` which will not need to have the arena for allocation.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message