Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of aleksey.ignatenko@gmail.com
 designates 64.233.184.235 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=d8abPQKzP2M+nHiEB4hd4rZqpnQPmkmNG3hLmqAb4TJEuAmj+X9mJQ8VwQ7S9mH9mxZqcuEwkSjOFtZXEMCOKGbBe/SXi0V5zbHLrZqrZEk39e3tOX7RAS7/32zUMyxpZp68w7Z9idK+yLjvRXWSEmb6LzpcXGpLiEq3O/jlnH0=
Message-ID: <5836de490701092119u4bd0815dqc344e564ecb4accd@mail.gmail.com>
Date: Wed, 10 Jan 2007 11:19:42 +0600
From: "Aleksey Ignatenko" <aleksey.ignatenko@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [drlvm] stress.Mix / MegaSpawn threading bug
In-Reply-To: <51d555c70701091014n6f6874ocdb08fdeb383213e@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_533_3069785.1168406382328"
References: <97EB9EC8-ECCD-4E9A-A165-F16310C37133@pobox.com>
	 <eo079l$9p3$1@sea.gmane.org>
	 <4dd1f3f00701090751w23e9e3b5pcef55055f201544e@mail.gmail.com>
	 <51d555c70701091014n6f6874ocdb08fdeb383213e@mail.gmail.com>

------=_Part_533_3069785.1168406382328
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On 1/10/07, Rana Dasgupta <rdasgupt@gmail.com> wrote:
>
> On 1/9/07, Weldon Washburn <weldonwjw@gmail.com> wrote:
> >
> > On 1/9/07, Gregory Shimansky <gshimansky@gmail.com> wrote:
> > >> I've tried to analyze MegaSpawn test on windows and here's what I
> found
> > >> out.
> > >>
> > >> OOME is thrown because process virtual size easily gets up to 2Gb.
> This
> > >> happens at about ~1.5k simultaneously running threads. I think it
> > >> happens because all of virtual process memory is mapped for thread
> > stacks.
> > >>
> >
> > >Good job!  I got the same sort of hunch when I looked at the source
> code
> > did
> > >not have enough time to pin down specifics.  The only guidance I
> > >found regarding what happens when too many threads are spawned is the
> > >following in the java.lang.Thread reference manual, "...specifying a
> > lower
> > >[stacksize] value may allow a greater number of threads to exist
> > >concurrently without throwing an OutOfMemoryError (or other internal
> > >error)."
> >
> > >I think what the above implies is that it is OK for the JVM to error
> and
> > >exit if the app tries to create too many threads.  If this is the case,
> > it
> > >sort of looks like we need to clean up the handling of malloc() errors
> so
> > >that the JVM can exit gracefully.
>
>
> I am not sure that we need to do something about this. The default initial
> stack size on Windows is 1M, and that is the recommended init size for
> real
> applications. The fact that our threads start with a larger intial stack
> mapped( default ) than RI is a design issue, it is not a bug. We could
> start
> with 2K and create many more threads! Exactly as Gregory points out,
> ultimately we will hit virtual memory limits and fail. The reason the RI
> seems to fail less is that the test ends before running out of virtual
> memory.On my 32 bit RHEL Linux box, RI fails almost every time with
> MegaSpawn, with an identical OOME error message and stack dump.
>
> We can catch the exception in the test and print a message. But I am not
> very sure what purpose that would serve. A resource exhaustion exception
> is
> a fatal exception and the process is hosed, no real app would be able to
> do
> anything more at this point. We should not use this test ( which is not a
> real app ) as guidance to tune the initial stack size. My suggestion is to
> lower the test duration so that we can create about a 1000( or whatever
> magic number ) threads at least. That is the stress condition we should
> test
> for.


Can we find such a "magic number" for test? The test is highly dependent on
PC configuration (size of available virtual memory), so someone with 256Mb
installed only could have OOME problems with lower number of threads. Looks
like the root problem is in incorrect handling of native memory exhaustion.

Best regards,
Aleksey.

Thanks,
> Rana
>
>

------=_Part_533_3069785.1168406382328--