incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From serkan özal <serkanoza...@hotmail.com>
Subject [PROPOSAL] - Java OffHeap Memory Pool
Date Tue, 16 Apr 2013 17:46:57 GMT
Project Name: Jillegal


1. Abstract:
GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts
objects at memory. If there are so many live objects, managing them by GC leads to overhead.
If objects can be allocated outside of GC, there will be no overhead for the application.
The session will go through the new method of creating and using object off-heap with no additional
serialization/deserialization or any other overheads. 


2. Proposal:
For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap
instead of heap. Not only object attributes are allocated at off-heap, but also object header
and metadata are allocated at off-heap. So while a reference to this object at off-heap is
being interpreted, JVM knows which class this object references to. You can get your off-heap
object from an object pool instead of with "new" operator. This object pool allocates a fixed
size memory region from off-heap and create empty objects with given class on there. These
empty objects can be created as lazy or eager. Another advantage off this technique is that
objects in pool are layout in memory as sequential. While accessing an object, its neighbour
objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses
are increased. On the other hand, freeing unused objects is responsibility of developer by
calling "free" method of object pool which means there is no dead object detection mechanism
like GC. Therefore, getting all objects from off-heap for whole application is dangerous and
not recommended. Because, this will cause memory leaks. In addition, this technique can be
combined with "Java Instrumentation API". With "@FromOffHeap" annotation, developer can sign
classes these must be allocated from off-heap instead of heap. With "Java Instrumentation
API", all "new" operators for signed classes with "@FromOffHeap" annotation can ben transformed
to code that allocates objects of signed class from off-heap via object pool. So developer
doesn't change all "new" keywords for getting from object pool in code. Instread of this,
just sign class with "@FromOffHeap" annotation and all "new" keywords transformed for getting
from object pool at class load time. This technique was used at a real time CDR processing
system for Turk Telekom. There were billions of objects were used. Managing these objects
by GC caused to performance overhead. For some most used classes, we allocated these objects
from off-heap instead of "new" keyword. After some processings on them (takes 4-5 hours),
we release these allocated memory regions to operating system by freeing them. Allocating
objects from off-heap pool helps us to gain significant execution time performance.


3. Rationale:
In general, off-heap pool implementations are implemented by serialization/deserialization
to allocated off-heap memory region via "ByteBuffer" class. But this technique leads to extra
execution overhead. Because while reading from an object, the target object must be created
by deserializing all primitive fields eagerly or only required fields on demand and while
writing to an object, the attribute has been set by application, must be deserialized to allocated
off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks
it. With my solution, all of these overheads are overcomed.


4. Initial Goals:
         * Allocating objects from off-heap and using them as normal on-heap Java object.
        * Allocating arrays for object types from off-heap and using them as normal on-heap
Java object arrays.         * Allocating arrays for primitive types from off-heap and using
them as normal on-heap Java primitive type arrays.         * Allocating strings from off-heap
and using them as normal on-heap strings.         * Implementing auto expandable off-heap
pool that expands when its delegated off-heap pool implementation is full.         * All features
must be supported for 32 bit and 64 bit JVM.         * All features must be supported for
Sun HotSpot JVM, Oracle JRockit, IBM J9.

5. Currently Implemented Features:

         * Allocating objects from off-heap and using them as normal on-heap Java object 
       * Allocating arrays for object types from off-heap and using them as normal on-heap
Java object array         * Allocating arrays for primitive types from off-heap and using
them as normal on-heap Java primitive type arrays         * Implementing auto expandable off-heap
pool that expands when its delegated off-heap pool implementation is full.         * All features
are supported for 32 bit and 64 bit JVM.         * All features are supported for Sun HotSpot
JVM, Oracle JRockit (IBM J9 support will be added).


6. Roadmap
         * Automatic detection and binding for complex off-heap objects will be implemented.
        * All allocated objects with new operator will be automatically allocated from off-heap
           without any changing in your code. Just annotate class whose instances must be
allocated from off-heap           with "@FromOffHeap" annotation. This feature can be implemented
with Java Instrumentation API by transforming           all "new" byte codes for specified
classes to a code block like "OffHeapPool.getObject(Class clazz)"
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message