harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rana Dasgupta" <rdasg...@gmail.com>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Mon, 05 Feb 2007 17:22:40 GMT
On 2/5/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
>
> Rana Dasgupta wrote:
> >> On 2/3/07, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> >>>
> >>
> >>> >Tim, the pinned memory is required by the API spec. We have to
> support
> >>> >it either in VM or API. (GC to support it doesn't necessarily mean to
> >>> >have a pinned space in GC managed heap.) It's probably good to
> >>> >distinct two different situations:
> >>>
> >>> >1. a native resource is associated with a Java object, and the
> >>> >resource can only be released when the object is strongly
> unreachable.
> >>> >In this case, the programmer who uses this class knows that a GC
> cycle
> >>> >can help to release the unused resource. Then in his code, he
> probably
> >>> >can directly invoke some VM interface to force a collection when the
> >>> >resource is run out. We need certain VM support that serve the
> >>> >purpose. Or the programmer just relies on the VM to provide sort of
> >>> >automatic resource management.
> >>>
> >>> >2. a native resource is associated with a Java object, but managed by
> >>> >the application; or the native resource is not associated with a Java
> >>> >object at all (e.g., completely managed in native libraries). In this
> >>> >case, the whole resource management burden is taken by the
> application
> >>> >itself. There is no requirement to VM.
> >>
> >>
> >> I agree. My contention is that management of most unmanaged resources
> come
> >> under this category. C# for example, recommends a Dispose() design
> pattern
> >> to release resources and the language ties it back to the finalizer. I
> >> would suggest that we focus on solving the unmanaged memory release
> problem
> >> which to me looks more critical because unlike threads, fd's etc, where
> >> mappings are usually 1x1 between the unmanaged resource and the
> >> corresponding java object, in this case, the api seems to allow tying
> very
> >> large chunks of memory to the bytebuffer without an explicit release
> api. I
> >> don't think that the JVM( except for specialized ones ) has the
> >> responsibility to automatically account for and manage all types of
> >> resources, but it is obliged to do automatic memory management as Xiao
> Feng
> >> point out below.
>
> >I don't see the distinction.  There is still a 1:1 between the Java
> >object and the native resource.  If you run out of file handles then it
> >is potentially as bad as running out of native memory, the disparity in
> >size between the Java resource and the native resource is immaterial
> >isn't it?


    There is  a distinction. One could directallocate only a few buffers and
exhaust the C heap ( if I understand the api correctly ) if the api
implementation is not eager about releasing back unused resources. One is
unlikely to hit a limit with a few outstanding fd's. I don't think that we
need to expand the scope of the problem to do all types of resource
accounting in the JVM or in an underlying platform layer. I don't see the
spec providing any basic guidance in this area, it falls in the category of
failsafe JVM's which is an interesting area, but IMHO beyond scope at this
point. My suggestion is that we treat this as a specific nio api
implementation problem for which we can think of adding some VM/GC support
if it helps. The extent of  the support is based on how widely this fails in
real usage scenarios. After all the doc seems to say that the directbuffer
is best chosen for long lived large buffers only, those that will need
infrequent collection, implying weak GC dependance.
    One way to do this could be to treat these as a special class of Java
objects with all allocation on the Java heap, as Xiao Feng suggests, and let
GC decide on handling the resource exhaustion. The second is for the api to
ask the GC some specific questions about unreachable object instances to
eagerly decide on deallocation. That requires a private interface. The third
is for the api implementation to do most of the housekeeping ( hence my
suggestion about max memory ) without relying on platform hooks, and
heuristically decide on when to invoke GC. This will fail sometimes, but
that may be OK. The solution can be made more efficient ( than finalizers
etc. ) by making the interface private. The last would be fore the api to
add platform hooks to help with its housekeeping. But I don't believe that
this is possible for this solution to be perfect( imagine several VM
instances running on a client machine ), and forces a dependency on a third
platform dependant piece, if it is to be kept independent of classlib and
VM.



> >>> The case for direct ByteBuffer belongs to 1. I am afraid certain VM
> >>> >support is necessary. The question is what and how.
> >>
> >>
> >> I am not clear on what's the best approach. It seems that the most
> general
> >> automatic solution would need to be based on reference counting, but
> >> that is not a small task.
>
> >Reference counting of the Java object I presume?  How would that work?\


I don't know of a good automatic way to do ref counting of the java object
without generating additional code by the jit when references are being
copied, etc. It is therefore not trivial to get right and has performance
impacts. It is a generalized way for the GC to answer questions about
instance lifetimes discussed above. Anyway, we should probably treat this as
a GC implementation issue.

>> I don't think that external monitoring threads etc. are
> >> very neat. There is no way to get them perfectly right in spite of the
> non
> >> trivial development needed to build them.  Xiao Feng suggested a
> >> feeDirect()
> >> method which could be a good way, but one of the api people would need
> to
> >> confirm if this addition is feasible.
>
> >We already have a Harmony-specific way to free the memory, which we can
> >use in implementing the class libraries themselves, but that does not
> >help our users who will be writing portable code.


This interface is not user visible. But I understand that it has some impact
on coupling the api implementation with VM's that don't support the private
interfaces. It makes those implementations somewhat less efficient.

>>>  Is it feasible to add an option to the VM specifying a maxDirectMemory,
> >>> and for the api implemententaion to keep an account of how much has
> been
> >>> allocated and freed ? And on being requested for a new allocation, if
> it
> >>> thinks necessary, invoke on a gc private interface to force cleanup?
>
> >That is one of the suggestions so far.  In the absence of a private
> >interface it would have to be System.gc() to detect that the Java object
> >is collectable, and run finalization/put the phantom ref on the queue --
> >if we had a private interface we could ask the VM if a specific object
> >is reachable (not sure how easy it is to answer that question in
> general).



Regards,
> Tim
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message