harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Markov" <mikhail.a.mar...@gmail.com>
Subject Re: [classlib] H-3148 (OOM when using native memory which is out of GC-control) again
Date Thu, 17 May 2007 14:48:25 GMT
I'd like to note here: I've made a lot of experiments and see no problem
here:
we're asking the system for *physical* memory availability (at least this is
what the patch is trying to do), while there are still enough virtual memory
left.
For example: on WinXP the threshold is exceeded after 10-20 iterations but
OOM really happened (if not call GC) only after 41 iterations.
That's why 64M + 95% was enough for me to smoothly run the test.

Thanks,
Mikhail

On 5/17/07, Leo Li <liyilei1979@gmail.com> wrote:
>
> Another problem is the concern for the threshold to invoke GC. One problem
> is that it is hardcoded. But of more importance is that the condition is
> too
> tough, for example, 512M on win32 platform as Alexey Varlamov has
> mentioned
> on the jira comments .
>
> I think one reason for that is when the threshold is met and we invoke
> System.gc(), but the thread is not blocked until gc completes. So user
> thread is still allocating native memory. Thus the threshold must be set
> so demanding in order to prevent out-of-memory before gc takes effect. And
> it also leads to performance problem, for one thing, gc is invoked too
> early. For another, several request for gc will be invoked during the gap
> from the first time memory threshold is hit till the gc takes effect at
> last.(Of course how many gc is actually invoked is decided by vm.)
>
> Although it seems impossible to get direct support from java public api,
> is it possible for our vm to provide a call that will surely invoke a new
> gc
> (or do nothing when a gc is now running )and let the caller blocked till
> current gc completes?
>
> If so, I believe the threshold will be much more acceptable and near the
> free memory demand for gc of VM. And I can try a self-adaptive strategy,
> like the working set management of os, to decide the threshold
> dynamically.
>
>
> Good luck!
>
> On 5/17/07, Leo Li <liyilei1979@gmail.com> wrote:
> >
> >  Excuse me, actually I have just found there is a bug about the patch on
> > windows platform. That is why the testcase runs slow. I will modify it
> now.
> >  Sorry for the inconvenience.
> >
> >
> >  On 5/17/07, Mikhail Markov <mikhail.a.markov@gmail.com> wrote:
> > >
> > > On 5/16/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > >
> > > > On 5/15/07, Mikhail Markov <mikhail.a.markov@gmail.com> wrote:
> > > > >
> > > > > Hi, all!
> > > > >
> > > > > I'd like to raise the problem with freeing native memory which is
> > > out of
> > > > > GC
> > > > > control again :-) (and
> > > > https://issues.apache.org/jira/browse/HARMONY-3148as
> > > > > one of it's demonstration).
> > > > > (See the previous round at
> > > > > http://thread.gmane.org/gmane.comp.java.harmony.devel/25768).
> > > > >
> > > > > Several people have added comments to the JIRA, but we need a
> > > general
> > > > > decision on the following question:
> > > > >
> > > > > Do we accept the way which was introduced by Leo's patch in H-3148
> (
> > > i.e.
> > > > > check if there are enough native memory available before
> allocating
> > > new
> > > > > one,
> > > > > and call System.gc() (or System.runFinalization()) if necessary)?
> > > > >
> > > > > I'm +1 for this method.
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I'm not sure whether I understand the problem correctly, but is it
> > > > possible
> > > > that vm onlys invokes gc when it fails to allocate, instead of
> > > checking
> > > > native memory every time before allocation which would may cause
> > > > performance
> > > > downgrade?
> > >
> > >
> > > Well, i've tried both approaches on the test described in H-3148 (100
> > > iterations of allocating 20Mb DirectByteBuffer-s) on my laptop with
> J9:
> > > 1) Pro-active memory monitoring (I've changed thresholds to 95% &
> 64M):
> > > the
> > > test works ok, I could do normal work in parallel
> > > 2) Recovering from OOM: the test works terribly slow, I couldn't do
> > > anything
> > > else as there was active disk swapping. On 42nd iteration the OOM was
> > > got,
> > > System.gc () was invoked and after that the memory allocation was
> > > retried. It
> > > was unsuccessfull and OOM was still got (perhaps there was not enough
> > > memory
> > > already even to proper run GC).
> > >
> > > So, even if second allocation in 2-nd method was successfull, i'd
> prefer
> > > the
> > > first one :-).
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > (Mark mentioned that he'd refactored the patch if he had time:-) - i'm
> > > ready
> > > > > to do this if he has no time.)
> > > > >
> > > > > Thanks,
> > > > > Mikhail
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew Zhang
> > > >
> > > > http://zhanghuangzhu.blogspot.com/
> > > >
> > >
> >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message