harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Volosyuk" <ivan.volos...@gmail.com>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Mon, 05 Feb 2007 00:54:35 GMT
On 2/4/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
> Leo Li wrote:
> > On 2/4/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
> >>
> >> Ivan Volosyuk wrote:
> >> > On 2/3/07, LvJimmy´╝îJing <firepure@gmail.com> wrote:
> >> >> 2007/2/2, Xiao-Feng Li <xiaofeng.li@gmail.com>:
> >> >> > Thank you, Jimmy and Leo, for the clear explanations.
> >> >> >
> >> >> > I think temporarily you can put the System.gc() there to workaround
> >> >> > this issue. Yes, this is only a workaround since we cannot rely
> >> on it
> >> >> > without explicit guarantee in its semantics.
> >> >> >
> >> >> > To really solve this problem, we might need a careful design.
Your
> >> >> > proposed approach looks like a possible solution. Have you
> >> thought to
> >> >> > solve the problem by using GC to manage the raw memory blocks?
> >> Without
> >> >> > deep understanding of the problem, I have some wild idea below.
> >> Please
> >> >> > see if it is reasonable or stupid. I will read the spec of
> >> ByteBuffer
> >> >> > to get more understanding.
> >> >> >
> >> >> > Since what the ByeBuffer needs are starting address and capacity,
it
> >> >> > doesn't really care if this piece of memory is in Java heap or
in
> >> >> > native runtime memory ( I assume.) We probably can provide some
> >> >> > special kind of virtual Java object that serves as raw memory
block
> >> to
> >> >> > nio. In this works, we need not monitor the native runtime memory
> >> >> > usage.
> >> >> >
> >> >>
> >> >> Ah, this solution sounds interesting to me, yes, if we malloc memory
> >> >> in heap then we no longer need to monitor the native memory. But we
> >> >> may need a kit of methods offer this heap-memory service:
> >> >> 1) a function of malloc_in_heap()
> >> >> 2) a function of free_heap_block()
> >> >> 3) heap monitor should record this malloc/free action
> >> >> 4) ...
> >> >> However I do think this may be a good solution, and it still depends
> >> >> on how easy can it be implemented. Note not only direct ByteBuffer
> >> >> require this native memory usage.
> >> >
> >> > I like this idea. I think we can express it in more general way. We
> >> > need native memory which is somehow related to java heap objects. When
> >> > calling garbage collection there is high possibility that some of this
> >> > memory will be freed. So here is my variant of this proposal:
> >> > 1. a function gc_native_malloc() which allocates native memory and
> >> > triggers GC on a certain threshold.
> >> > 2. a fenction gc_native_free() which deallocates native memory and
> >> > updates counter for the used space.
> >> > This memory may be allocated in general native heap but it should be
> >> > accounted by GC. It is not even required that it should be the GC who
> >> > implements this functionality, just some code which will can call GC
> >> > and able to allocate native memory.
> >> >
> >> > One drawback of this approach that we should know all the places where
> >> > we have a relation between java heap object and native memory. Some
> >> > custom native library may disobey this rule.
> >>
> >> (Sorry for repeating this through gmane. Gmail today is acting weird and
> >> doesn't accept my emails through SSMPT)
> >>
> >> I see no difference in the approach I've suggested already. If we have
> >> to take care about all the native resource allocation inside of classlib
> >> APIs, then there is no difference which code calls GC, be it
> >> gc_native_malloc, or the API native code like this:
> >>
> >> void *p = malloc(size);
> >> if (NULL == p)
> >> {
> >>   gc_force_gc();
> >>   p = malloc(size);
> >>   if (NULL == p)
> >>   {
> >>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >>       return;
> >>   }
> >> }
> >
> >
> > But I am worried whether it is too late to force gc when memory allocation
> > fails. It will take some time to complete the release of resources, for
> > example, in finalizer() and this period of time is not predictable. Thus
> > the
> > malloc will still fails...:(
> > Futhermore, if we wait till memory allocation fails then to force gc, it
> > might be possible that more than one thread will encounter such a problem
> > and force gc. I am not sure whether it will lead to problem in vm.
> > So, I think it may be wiser to trigger a gc when free memory ratio is
> > low and before the failure in malloc actually occurs.
>
> Yes, I shouldn't have called the above code a "solution". I just wanted
> to point to Ivan that there is no difference with taking care about
> memory allocation in GC code or in classlib code.

There only difference here is conceptual. If we request a resource
from GC via gc_malloc_native() or something like that, we may be sure
that GC will do the job of tracking that resources and will try to do
something when we go out of free space.

>
> You've found another problem in such approach, just running GC is not
> enough in this case because finalization may not be done in time to free
> needed native resources.
>
> Speaking in DRLVM terms, the above code could also contain call to
> vm_run_pending_finalizers, but it is still not a solution for
> multithreaded case.
>
> --
> Gregory
>
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Mime
View raw message