Mailing-List: contact dev-help@river.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@river.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: Peter <jini@zeus.net.au>
Reply-To: Peter <jini@zeus.net.au>
To: dev@river.apache.org, Simon Roberts <simon@dancingcloudservices.com>
Subject: Re: JERI Scalability Testing
References: <5370DDF0.2000302@zeus.net.au>
	  <CADsQA7iNN_Npp9ca1ZBeCONiEm6+sh3z1RU7p0qWiwUxng4_pA@mail.gmail.com>
In-Reply-To: 
 <CADsQA7iNN_Npp9ca1ZBeCONiEm6+sh3z1RU7p0qWiwUxng4_pA@mail.gmail.com>
Content-Type: multipart/alternative; boundary="=-Fgd+pRLWAZren10Zkn3y"
Date: Tue, 13 May 2014 09:17:41 +1000
Message-Id: <1399936661.23111.5.camel@Nokia-N900>
Mime-Version: 1.0

--=-Fgd+pRLWAZren10Zkn3y
Content-Type: text/plain; charset=utf-8
Content-ID: <1399936661.23111.2.camel@Nokia-N900>
Content-Transfer-Encoding: 8bit

Hi Simon,

http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/net/jini/loader/ClassLoading.java?view=markup

During stress test profiling I found Class.forName to be heavily contended (the method with a ClassLoader parameter).

Uncontended synchronization is very fast, so rather than employ parallel ClassLoading (which requires a lock for every class and isn't part of the jvm or lang specs), I decided to try thread confinement instead (for each ClassLoader).  Before I found class loading to be a bottle neck, I had to fix a number of other throughput bottlenecks however:

1. Security Policy provider would cause contention while it performed dns calls.  Dns calls are still made, but far fewer and concurrent security checks are now non blocking.  The new policy provider takes advantage of immutability, how often do you change your policy files?
2. SecureClassLoader uses CodeSource in a loader map as keys, causing multiple dns lookup calls, fixed that with a RFC3986 compliant URI class to replace URL based keys in maps.
3. PreferredClassLoader also used URL's as keys in maps.
4. Exising thread pools didn't take advantage of concurrent utilities and TaskManager performed very poorly when it's task queue became large, since dependant tasks had to synchronize and iterate over the whole queue to find preceeding tasks.

All fixed now, just need to reduce network traffic as Sockets are now the bottleneck.

After I finish with latent bugs, I'll investigate using a provider to plug in various serialization frameworks that are available now.

That and codebase provisioning should make River really sing.

The stress tests were devised by the original Jini team, they're probably the closest thing I've got to deployment scenario's.

Regards,

Peter.
----- Original message -----
> Peter,
> 
> With apologies that this it totally off-topic, but you made a comment
> here that I'd really like to understand better:
> 
>       * ClassLoading is thread confined for each classloader to avoid
> > contention.
> 
> 
> Without impinging on your time unduly, are you able to point me at
> something that would allow me to discover what this is about? It sounds
> interesting, but I have no idea how this would work.
> 
> And, I quite understand if you remain silent :)
> 
> Cheers,
> Simon


--=-Fgd+pRLWAZren10Zkn3y--