harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weldon Washburn" <weldon...@gmail.com>
Subject Re: [drlvm] stress.Mix / MegaSpawn threading bug
Date Wed, 10 Jan 2007 06:20:43 GMT
On 1/9/07, Aleksey Ignatenko <aleksey.ignatenko@gmail.com> wrote:
>
> On 1/10/07, Rana Dasgupta <rdasgupt@gmail.com> wrote:
> >
> > On 1/9/07, Weldon Washburn <weldonwjw@gmail.com> wrote:
> > >
> > > On 1/9/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
> > > >> I've tried to analyze MegaSpawn test on windows and here's what I
> > found
> > > >> out.
> > > >>
> > > >> OOME is thrown because process virtual size easily gets up to 2Gb.
> > This
> > > >> happens at about ~1.5k simultaneously running threads. I think it
> > > >> happens because all of virtual process memory is mapped for thread
> > > stacks.
> > > >>
> > >
> > > >Good job!  I got the same sort of hunch when I looked at the source
> > code
> > > did
> > > >not have enough time to pin down specifics.  The only guidance I
> > > >found regarding what happens when too many threads are spawned is the
> > > >following in the java.lang.Thread reference manual, "...specifying a
> > > lower
> > > >[stacksize] value may allow a greater number of threads to exist
> > > >concurrently without throwing an OutOfMemoryError (or other internal
> > > >error)."
> > >
> > > >I think what the above implies is that it is OK for the JVM to error
> > and
> > > >exit if the app tries to create too many threads.  If this is the
> case,
> > > it
> > > >sort of looks like we need to clean up the handling of malloc()
> errors
> > so
> > > >that the JVM can exit gracefully.
> >
> >
> > I am not sure that we need to do something about this. The default
> initial
> > stack size on Windows is 1M, and that is the recommended init size for
> > real
> > applications. The fact that our threads start with a larger intial stack
> > mapped( default ) than RI is a design issue, it is not a bug. We could
> > start
> > with 2K and create many more threads! Exactly as Gregory points out,
> > ultimately we will hit virtual memory limits and fail. The reason the RI
> > seems to fail less is that the test ends before running out of virtual
> > memory.On my 32 bit RHEL Linux box, RI fails almost every time with
> > MegaSpawn, with an identical OOME error message and stack dump.
> >
> > We can catch the exception in the test and print a message. But I am not
> > very sure what purpose that would serve. A resource exhaustion exception
> > is
> > a fatal exception and the process is hosed, no real app would be able to
> > do
> > anything more at this point. We should not use this test ( which is not
> a
> > real app ) as guidance to tune the initial stack size. My suggestion is
> to
> > lower the test duration so that we can create about a 1000( or whatever
> > magic number ) threads at least. That is the stress condition we should
> > test
> > for.
>
>
> Can we find such a "magic number" for test? The test is highly dependent
> on
> PC configuration (size of available virtual memory), so someone with 256Mb
> installed only could have OOME problems with lower number of threads.


I agree.



 Looks
> like the root problem is in incorrect handling of native memory
> exhaustion.


I agree.  I put stress.Mix on the exclusion list for now.  And committed the
modified exclusion list.

Best regards,
> Aleksey.
>
> Thanks,
> > Rana
> >
> >
>
>


-- 
Weldon Washburn
Intel Enterprise Solutions Software Division

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message