Mailing-List: contact esme-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: esme-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of feeder.of.the.bears@gmail.com
 designates 209.85.211.182 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=KjG7f4xBAW3IyXhAlx3GvdmzDehSkB5J0RSGI7qcOzClvHQYA+yXCBM5hyvM+fmQQl
         NQLvHPlv0fc0Hp1c44prn0R2XrOS/BnMseZL98zm3W96Fsd2bT0EVNpd2N10GrQyTFUn
         PRkvyRfjWmp/CzbkJVkDSvVCYqI7Ij5JxKWbI=
MIME-Version: 1.0
In-Reply-To: <771905290911171325p160b0e4ci3fad2465322158e2@mail.gmail.com>
References: <771905290911171325p160b0e4ci3fad2465322158e2@mail.gmail.com>
Date: Tue, 17 Nov 2009 13:36:03 -0800
Message-ID: <cdbebedf0911171336gc9b5446s5d13ec81ab0d5a67@mail.gmail.com>
Subject: Re: First mass user tests results
From: David Pollak <feeder.of.the.bears@gmail.com>
To: esme-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=001636164bdbc0999c047897e908

--001636164bdbc0999c047897e908
Content-Type: text/plain; charset=UTF-8

Markus,

Interesting information.

I found a bug in the Lift Actors (fix checked into master this morning)
where the Actor thread pool would grow unbounded.  Given the amount of
message passing in ESME, I think threads were being created rather queuing
messages.  I'd be interested to see if a build of ESME made with the current
SNAPSHOT of Lift exhibits the same thread explosion issue.

Also, I don't think the attachments made it through.

Thanks,

David

On Tue, Nov 17, 2009 at 1:25 PM, Markus Kohler <markus.kohler@gmail.com>wrote:

> Hi all,
> Yesterday night I finally got some tests running.
> I still focused on single threaded (serial) tests using Selenium RC Java to
> control one firefox browser.
> *Test 1, Creating Users*
> The first test would create the 300+x users, using a CSV file I generated
> from my twitter followers. The test script enters User data including the
> url for the avatar and then logs out. Basically that means that during the
> test only one user is logged on at any  given point in time. Sorry didn't
> make any screenshots of the Stax monitor. Learned in the meantime this would
> have been a good idea.
> The number of threads went up to 130, which I find  surprising, given that
> there were no users on the system in parallel.
>
> *Test2, Logon each user*
> In the second test I logon each user and do not logout afterwards. The idea
> was to see what the memory overhead of one user is.  I achieved this with
> one browser by clearing the cookies after the user has logged on.
> The memory_allUsers attachment shows that the number of threads increased
> dramatically beyond 1000.
>
> The memory also went up, but this is not at all an issue atm. Compared to
> what I've seen so far at Enterprise apps it's very low!
>
> After the test was run, I tried with one browser whether everything would
> work still fine. This caused an unexpected behavior of the server. See
> cpu_allUsers and memory_allUsers2 attachments.
> The system load went up dramatically and stayed there for while. When
> entering a message, this message would appear only very slowly or not all in
> the users timeline. The number of threads would go down after a while, but
> there was a second peak.Not sure where it came from.
>
> What's also interesting is that the number of classes grew overtime.
> I would assume that full GC's where running so they should have been
> reclaimed, if they would have been only of temporary nature.
>
> Note that Stax seems to run on Tomcat 6 without the async/Comet support of
> the Servlet 3.0 API.
> The will wait for 7.0 to support that.
>
> As soon as I have some time, I will rerun the test on my local machine,
> where I have more tools to check what is going on.
> I will also first run it on Jetty to see whether it performs better.
>
> Still I would assume that NW CE will show the same issues and sooner or
> later we will have to figure out the root cause.
>
>
> Greetings,
> Markus
>
>
>
>
>
>


-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

--001636164bdbc0999c047897e908--