harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin Garner <robin.gar...@anu.edu.au>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Fri, 09 Feb 2007 02:38:17 GMT
Tim Ellison wrote:
> Rana Dasgupta wrote:
>> On 2/5/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
>>>> I don't see the distinction.  There is still a 1:1 between the Java
>>>> object and the native resource.  If you run out of file handles then it
>>>> is potentially as bad as running out of native memory, the disparity in
>>>> size between the Java resource and the native resource is immaterial
>>>> isn't it?
>>
>>    There is  a distinction. One could directallocate only a few buffers and
>> exhaust the C heap ( if I understand the api correctly ) if the api
>> implementation is not eager about releasing back unused resources. One is
>> unlikely to hit a limit with a few outstanding fd's.
> 
> My point was that we could potentially run out of any OS resource being
> held onto by a Java object before running out of Java heap for those
> objects.  Maybe file handles was a poor example.

I think the file handle mention was my fault, but it came from a 
concrete example: Dacapo xalan in the 2006-10 release, which would fail 
for exactly this reason if the heap was large enough.

>> I don't think that we need to expand the scope of the problem to do 
>> all types of resource accounting in the JVM or in an underlying 
>> platform layer. I don't see the spec providing any basic guidance in 
>> this area, it falls in the category of failsafe JVM's which is an
>> interesting area, but IMHO beyond scope at this point. My suggestion
>> is that we treat this as a specific nio api implementation problem
>> for which we can think of adding some VM/GC support if it helps.
> 
> We may have to disagree here then, I think we should indeed be
> considering a general solution for detecting early when these Java
> 'handle' objects are collectable -- then we can apply it to direct
> buffers or other handle types.

My point was that by adding a single GC -> VM interface function, all 
types of native resources could be addressed in a simple extensible way. 
  The subsequent policy and mechanism decisions then become an ongoing 
design process for the VM and classlib native implementers, but once 
this interface function is added, GC can support freeing of native 
resources by whatever mechanisms the VM chooses to implement.

>> The extent of the support is based on how widely this fails in real
>> usage scenarios. After all the doc seems to say that the directbuffer
>> is best chosen for long lived large buffers only, those that will
>> need infrequent collection, implying weak GC dependance.
> 
> Yep, while we have a test case that demonstrates the difference between
> Harmony and other implementations, a failing application would be more
> compelling.
> 
>> One way to do this could be to treat these as a special class of Java
>> objects with all allocation on the Java heap, as Xiao Feng suggests, and
>> let GC decide on handling the resource exhaustion.
> 
> I suggest that would have it's own problems though.  The memory would
> have to be contiguous and pinned, and may be large.  We could end up
> with a fragmented heap quite quickly.

I strongly disagree with special-casing any of these resources.  To me, 
supporting pinned allocation for direct buffers is orthogonal to 
providing a way for the VM to request GC when certain native resources 
are at risk of depletion.

>> The second is for the api to ask the GC some specific questions about
>> unreachable object instances to eagerly decide on deallocation. That
>> requires a private interface.
> 
> right, which is an optimization of the suggestion of using the public
> System.gc() API

Surely a well-written Java application releases native resources either 
explicitly or through a reference type or finalizer ?  Yes, it's nice to 
tolerate badly written apps but is that the first-order concern here ?

>> The third is for the api implementation to do most of the
>> housekeeping ( hence my suggestion about max memory ) without relying
>> on platform hooks, and heuristically decide on when to invoke GC.
>> This will fail sometimes, but that may be OK. The solution can be
>> made more efficient ( than finalizers etc. ) by making the interface
>> private.
> 
> What heuristic would you suggest beyond the malloc failure?

% (native)heap usage, with some kind of 'damping' heuristic to prevent 
thrashing.

>> The last would be fore the api to add platform hooks to help with its
>> housekeeping. But I don't believe that this is possible for this
>> solution to be perfect( imagine several VM instances running on a
>> client machine ), and forces a dependency on a third platform
>> dependant piece, if it is to be kept independent of classlib and VM.
> 
> Which 'platform hooks' do you mean? (not sure if you meant they are in
> Harmony at the moment or are speaking hypothetically).
> 
> Regards,
> Tim


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Mime
View raw message