harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Volosyuk" <ivan.volos...@gmail.com>
Subject Re: [VM]How to trigue GC while free native memory is low.
Date Mon, 05 Feb 2007 03:24:31 GMT
On 2/5/07, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> On 2/5/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> >
> > On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
> >
> > > On 2/5/07, Ivan Volosyuk <ivan.volosyuk@gmail.com> wrote:
> > >> On 2/4/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> > >> >
> > >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> > >> >
> > >> > > Leo Li wrote:
> > >> > >> On 2/4/07, Gregory Shimansky <gshimansky@gmail.com>
wrote:
> > >> > >>>
> > >> > >>> I see no difference in the approach I've suggested already.
> > >> If we
> > >> > >>> have
> > >> > >>> to take care about all the native resource allocation
inside of
> > >> > >>> classlib
> > >> > >>> APIs, then there is no difference which code calls GC,
be it
> > >> > >>> gc_native_malloc, or the API native code like this:
> > >> > >>>
> > >> > >>> void *p = malloc(size);
> > >> > >>> if (NULL == p)
> > >> > >>> {
> > >> > >>>   gc_force_gc();
> > >> > >>>   p = malloc(size);
> > >> > >>>   if (NULL == p)
> > >> > >>>   {
> > >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> > >> > >>>       return;
> > >> > >>>   }
> > >> > >>> }
> > >> > >>
> > >> > >>
> > >> > >> But I am worried whether it is too late to force gc when
memory
> > >> > >> allocation
> > >> > >> fails. It will take some time to complete the release of
> > >> > >> resources, for
> > >> > >> example, in finalizer() and this period of time is not
> > >> > >> predictable. Thus
> > >> > >> the
> > >> > >> malloc will still fails...:(
> > >> > >> Futhermore, if we wait till memory allocation fails then
to
> > >> force
> > >> > >> gc, it
> > >> > >> might be possible that more than one thread will encounter
> > >> such a
> > >> > >> problem
> > >> > >> and force gc. I am not sure whether it will lead to problem
> > >> in vm.
> > >> > >> So, I think it may be wiser to trigger a gc when free memory
> > >> ratio is
> > >> > >> low and before the failure in malloc actually occurs.
> > >> > >
> > >> > > Yes, I shouldn't have called the above code a "solution". I just
> > >> > > wanted
> > >> > > to point to Ivan that there is no difference with taking care
> > >> about
> > >> > > memory allocation in GC code or in classlib code.
> > >> > >
> > >> > > You've found another problem in such approach, just running GC
> > >> is not
> > >> > > enough in this case because finalization may not be done in
> > >> time to
> > >> > > free
> > >> > > needed native resources.
> > >> > >
> > >> > > Speaking in DRLVM terms, the above code could also contain
> > >> call to
> > >> > > vm_run_pending_finalizers, but it is still not a solution for
> > >> > > multithreaded case.
> > >> >
> > >> > I'm getting confused on where people think that this problem
> > >> needs to
> > >> > be managed.  Am I misunderstanding that there should be some
> > >> > awareness of this in the classlib or VM natives?  Seems like it can
> > >> > be localized.
> > >> >
> > >> > I think that if whoever calls malloc() finds that the system is out
> > >> > of memory, then it's too late.  I always thought that VMs manage
> > >> > their conceptual native "heap" (all native resources consumed from
> > >> > OS, including as Robin mentioned, filehandles) in a proactive
> > >> manner,
> > >> > like they do their java heap.  it's clear now that they don't.
> > >> >
> > >> > So when a caller to malloc() gets a null, it should simply throw an
> > >> > OOM, assuming that the runtime environment did all it could to
> > >> > prevent that.
> > >> >
> > >> > Can this be done through portlib, letting the VMs implementation of
> > >> > it manage to whatever degree of sophistication is warranted?
> > >>
> > >> Actually, the portlib is the lowest layer we rely on. If we can have
> > >> here a callback to VM's GC we can place the code in the portlib. I
> > >> would also distinguish memory allocations which associated with java
> > >> objects and that which doesn't. No need to account resources which
> > >> will not be freed after garbage collection.
> > >>
> > >> As was mentioned we have different types of critical system
> > >> resources.
> > >> It makes sense to make a list of such resources:
> > >>    malloc'ed memory
> > >>    mmap'ed memory
> > >>    shared memory (?)
> > >>    file descriptors (files, sockets)
> > >>    XWin / GDI resources (?)
> > >>    more?
> > >>
> > >> What would be the accounting strategy for the resources? We could
> > >> allocate them via some functions in this facility or we could only
> > >> account them here and call GC using some buildin criteria.
> > >
> > > More thoughts.
> > >
> > > It makes sense that allocation or accounting routine will block for
> > > some time until finalizers path completes as Robin suggested.
> >
> > That's not obvious to me.  Without thinking too hard about it, I'd
> > rather get an OOM than block.  For example, suppose you had some kind
> > of transaction processing system with some SLA.  When processing a
> > transaction, a OOM would at least give me a chance to fail/rollback
> > and let the transaction be tried again on another machine, whereas a
> > block seems I lose any decision-making control as an app.
> >
>
> I tend to think the same thing. That's why I wonder why the API has
> mallocDirect but no freeDirect. If memory management is a special
> exception (since VM has certain memory management facility like GC),
> I'd wish other native resources have their respective management APIs
> for application to use, or the application has to reply on VM's
> disposal.

FreeDirect is quite implementable. It is strange that it is not exist
in API. May be it is done this way to be conceptually the same as
ordinary java objects.

-- 
Ivan
Intel Enterprise Solutions Software Division

Mime
View raw message