I think we need to use load injectors and SLAMD with this thing.  Really having the clients on the same machine
is causing issues.  I highly recommend setting up some other machines as client injectors.  I can do this with
some machines.
Also Emmanuel we need to use SLAMD instead of inventing our own means to measure performance.  The SUN
guys really put a lot of work into SLAMD and we should use it.  Perhaps we can build our own custom tests if need
be.  SLAMD should be the default tool we use otherwise the measurments and techniques used are questionable.
If we use matt's machine as the server I can setup several machines on the internet to be used as the load
injectors.  Yes this is a bit crazy across the open internet but if we can generate enough load with a few
machines across several sites we should be able to peak the server.

On 3/27/07, Emmanuel Lecharny <elecharny@gmail.com> wrote:
Hi Matt,

I wasn't sure you subscribed, sorry for the duplicated mails :(

On 3/27/07, Matt Hogstrom < matt@hogstrom.org > wrote:
I subscribed to your list so you won't have to copy me.

I'd do a couple of things.  The system is Hyperthreaded so you have
"16" CPUs.  I'll reboot the system and go back to 8 real cores just
to make sure hyperthreading isn't a cause.  I'll do that tonight if
no one is on the system.

Ok, np. I don't exactly know what 'tonite' means in UTC, FYI, I'm UTC + 2. (this mail has been sent at 09h05UTC)
I'm not sure that HyperThreading could be the cause of bad performance I'm having...

Next, I've used the IBM VM for this kind of analysis as it has a Java
Lock monitor built in that will help to identify hot locks that might
be causing the scaling problem.

I will install IBM JVM today. I have tested the server with IBM, SUN and BEA jvms last year, and it was my intention to do so.

The next thing we can do is I have a hardware profiler on the system
that is similar to VTune that is most excellent at tracking CPU, time
in method, etc.  I need to recompile the kernel so this will take a
few days.

Oh, that would be great ! I feel like been totally blind when doing perf tests, just guessing about what could be the bottlenecks...

So if you want to take this baby for another spin around the block,
let me know.  I'll get some instructions on setting up the lock monitor.

Yeah, sure. I'm working full time on ADS for a little while, so anytime you want will be ok for me.

Nice to know my basement is getting warmed for good reason :)

yeah, I'm not sure it helps figth against global warming...At least, it's not using gazoline to run !


Emmanuel Lécharny