harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Volosyuk" <ivan.volos...@gmail.com>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Mon, 05 Feb 2007 02:41:15 GMT
On 2/5/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
> On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
>
> > On 2/5/07, Ivan Volosyuk <ivan.volosyuk@gmail.com> wrote:
> >> On 2/4/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> >> >
> >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> >> >
> >> > > Leo Li wrote:
> >> > >> On 2/4/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
> >> > >>>
> >> > >>> I see no difference in the approach I've suggested already.
> >> If we
> >> > >>> have
> >> > >>> to take care about all the native resource allocation inside
of
> >> > >>> classlib
> >> > >>> APIs, then there is no difference which code calls GC, be
it
> >> > >>> gc_native_malloc, or the API native code like this:
> >> > >>>
> >> > >>> void *p = malloc(size);
> >> > >>> if (NULL == p)
> >> > >>> {
> >> > >>>   gc_force_gc();
> >> > >>>   p = malloc(size);
> >> > >>>   if (NULL == p)
> >> > >>>   {
> >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >> > >>>       return;
> >> > >>>   }
> >> > >>> }
> >> > >>
> >> > >>
> >> > >> But I am worried whether it is too late to force gc when memory
> >> > >> allocation
> >> > >> fails. It will take some time to complete the release of
> >> > >> resources, for
> >> > >> example, in finalizer() and this period of time is not
> >> > >> predictable. Thus
> >> > >> the
> >> > >> malloc will still fails...:(
> >> > >> Futhermore, if we wait till memory allocation fails then to
> >> force
> >> > >> gc, it
> >> > >> might be possible that more than one thread will encounter
> >> such a
> >> > >> problem
> >> > >> and force gc. I am not sure whether it will lead to problem
> >> in vm.
> >> > >> So, I think it may be wiser to trigger a gc when free memory
> >> ratio is
> >> > >> low and before the failure in malloc actually occurs.
> >> > >
> >> > > Yes, I shouldn't have called the above code a "solution". I just
> >> > > wanted
> >> > > to point to Ivan that there is no difference with taking care
> >> about
> >> > > memory allocation in GC code or in classlib code.
> >> > >
> >> > > You've found another problem in such approach, just running GC
> >> is not
> >> > > enough in this case because finalization may not be done in
> >> time to
> >> > > free
> >> > > needed native resources.
> >> > >
> >> > > Speaking in DRLVM terms, the above code could also contain
> >> call to
> >> > > vm_run_pending_finalizers, but it is still not a solution for
> >> > > multithreaded case.
> >> >
> >> > I'm getting confused on where people think that this problem
> >> needs to
> >> > be managed.  Am I misunderstanding that there should be some
> >> > awareness of this in the classlib or VM natives?  Seems like it can
> >> > be localized.
> >> >
> >> > I think that if whoever calls malloc() finds that the system is out
> >> > of memory, then it's too late.  I always thought that VMs manage
> >> > their conceptual native "heap" (all native resources consumed from
> >> > OS, including as Robin mentioned, filehandles) in a proactive
> >> manner,
> >> > like they do their java heap.  it's clear now that they don't.
> >> >
> >> > So when a caller to malloc() gets a null, it should simply throw an
> >> > OOM, assuming that the runtime environment did all it could to
> >> > prevent that.
> >> >
> >> > Can this be done through portlib, letting the VMs implementation of
> >> > it manage to whatever degree of sophistication is warranted?
> >>
> >> Actually, the portlib is the lowest layer we rely on. If we can have
> >> here a callback to VM's GC we can place the code in the portlib. I
> >> would also distinguish memory allocations which associated with java
> >> objects and that which doesn't. No need to account resources which
> >> will not be freed after garbage collection.
> >>
> >> As was mentioned we have different types of critical system
> >> resources.
> >> It makes sense to make a list of such resources:
> >>    malloc'ed memory
> >>    mmap'ed memory
> >>    shared memory (?)
> >>    file descriptors (files, sockets)
> >>    XWin / GDI resources (?)
> >>    more?
> >>
> >> What would be the accounting strategy for the resources? We could
> >> allocate them via some functions in this facility or we could only
> >> account them here and call GC using some buildin criteria.
> >
> > More thoughts.
> >
> > It makes sense that allocation or accounting routine will block for
> > some time until finalizers path completes as Robin suggested.
>
> That's not obvious to me.  Without thinking too hard about it, I'd
> rather get an OOM than block.  For example, suppose you had some kind
> of transaction processing system with some SLA.  When processing a
> transaction, a OOM would at least give me a chance to fail/rollback
> and let the transaction be tried again on another machine, whereas a
> block seems I lose any decision-making control as an app.

It makes sense. (After thinking hard) I have come to conclusion that
we just need to account the allocation rate of system resources and
after certain thresholds for each resource type trigger a GC. It will
help us to keep the resources at some specific level depending on
usage pattern and in case of abuse will lead to out of resource
condition. Quite simple solution and I have already saw it done in
DRLVM's thread manager implementation.

-- 
Ivan
Intel Enterprise Solutions Software Division

Mime
View raw message