harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gregory Shimansky <gshiman...@gmail.com>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Sun, 04 Feb 2007 13:41:45 GMT
Leo Li wrote:
> On 2/4/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
>>
>> Ivan Volosyuk wrote:
>> > On 2/3/07, LvJimmy´╝îJing <firepure@gmail.com> wrote:
>> >> 2007/2/2, Xiao-Feng Li <xiaofeng.li@gmail.com>:
>> >> > Thank you, Jimmy and Leo, for the clear explanations.
>> >> >
>> >> > I think temporarily you can put the System.gc() there to workaround
>> >> > this issue. Yes, this is only a workaround since we cannot rely 
>> on it
>> >> > without explicit guarantee in its semantics.
>> >> >
>> >> > To really solve this problem, we might need a careful design. Your
>> >> > proposed approach looks like a possible solution. Have you 
>> thought to
>> >> > solve the problem by using GC to manage the raw memory blocks?
>> Without
>> >> > deep understanding of the problem, I have some wild idea below.
>> Please
>> >> > see if it is reasonable or stupid. I will read the spec of 
>> ByteBuffer
>> >> > to get more understanding.
>> >> >
>> >> > Since what the ByeBuffer needs are starting address and capacity, it
>> >> > doesn't really care if this piece of memory is in Java heap or in
>> >> > native runtime memory ( I assume.) We probably can provide some
>> >> > special kind of virtual Java object that serves as raw memory block
>> to
>> >> > nio. In this works, we need not monitor the native runtime memory
>> >> > usage.
>> >> >
>> >>
>> >> Ah, this solution sounds interesting to me, yes, if we malloc memory
>> >> in heap then we no longer need to monitor the native memory. But we
>> >> may need a kit of methods offer this heap-memory service:
>> >> 1) a function of malloc_in_heap()
>> >> 2) a function of free_heap_block()
>> >> 3) heap monitor should record this malloc/free action
>> >> 4) ...
>> >> However I do think this may be a good solution, and it still depends
>> >> on how easy can it be implemented. Note not only direct ByteBuffer
>> >> require this native memory usage.
>> >
>> > I like this idea. I think we can express it in more general way. We
>> > need native memory which is somehow related to java heap objects. When
>> > calling garbage collection there is high possibility that some of this
>> > memory will be freed. So here is my variant of this proposal:
>> > 1. a function gc_native_malloc() which allocates native memory and
>> > triggers GC on a certain threshold.
>> > 2. a fenction gc_native_free() which deallocates native memory and
>> > updates counter for the used space.
>> > This memory may be allocated in general native heap but it should be
>> > accounted by GC. It is not even required that it should be the GC who
>> > implements this functionality, just some code which will can call GC
>> > and able to allocate native memory.
>> >
>> > One drawback of this approach that we should know all the places where
>> > we have a relation between java heap object and native memory. Some
>> > custom native library may disobey this rule.
>>
>> (Sorry for repeating this through gmane. Gmail today is acting weird and
>> doesn't accept my emails through SSMPT)
>>
>> I see no difference in the approach I've suggested already. If we have
>> to take care about all the native resource allocation inside of classlib
>> APIs, then there is no difference which code calls GC, be it
>> gc_native_malloc, or the API native code like this:
>>
>> void *p = malloc(size);
>> if (NULL == p)
>> {
>>   gc_force_gc();
>>   p = malloc(size);
>>   if (NULL == p)
>>   {
>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>>       return;
>>   }
>> }
> 
> 
> But I am worried whether it is too late to force gc when memory allocation
> fails. It will take some time to complete the release of resources, for
> example, in finalizer() and this period of time is not predictable. Thus 
> the
> malloc will still fails...:(
> Futhermore, if we wait till memory allocation fails then to force gc, it
> might be possible that more than one thread will encounter such a problem
> and force gc. I am not sure whether it will lead to problem in vm.
> So, I think it may be wiser to trigger a gc when free memory ratio is
> low and before the failure in malloc actually occurs.

Yes, I shouldn't have called the above code a "solution". I just wanted
to point to Ivan that there is no difference with taking care about
memory allocation in GC code or in classlib code.

You've found another problem in such approach, just running GC is not
enough in this case because finalization may not be done in time to free
needed native resources.

Speaking in DRLVM terms, the above code could also contain call to
vm_run_pending_finalizers, but it is still not a solution for
multithreaded case.

-- 
Gregory


Mime
View raw message