river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Help needed with concurrency bug
Date Wed, 28 Dec 2011 23:56:09 GMT
Spot on Dan, much appreciated, good to see Brian still watches the list 
too ;)

The policy providers are built for correctness & concurrency, first & 
foremost, once we've got that licked, we can have a look at performance.

N.B. Looking at some of the terrible hashCode and equals implementations 
in Permission classes gives me more confidence the PermissionComparator 
is the right decision, with the comparison being made on getClass(), 
getName() and getActions()

For example, not only does the SocketPermission hashCode possibly cause 
a dns lookup and generate a different hash for an IP address equal to a 
domain name based address, but is solely based on getName(), so 
identical hashes will be generated when actions differ and the 
Permission's don't imply or equal each other.  The only catch, if 
someone creates a Permission implementation with state that influences 
equality outside of name and actions... I haven't found any such 
implementations yet, everything appears to fit into the name or name & 
actions categories, this seems to be a general restriction imposed by 
java's policy parser.

We've already got some important optimisations:

   1. URIGrant is based on URI, not CodeSource or URL, so it won't cause
      a DNS lookup.
   2. URIGrant unlike CodeSource.implies, doesn't require disk access
      when normalising a URI.

The following optimisations are yet to be implemented:

   1. Order and create SocketPermissionCollection on demand to reduce
      DNS lookup (after failing a comparator based check for an exact
   2. OR reimplement SocketPermission.
   3. PermissionGrant is an interface that represents a grant statement
      in a policy file, implementations must be immutable, the current
      DynamicPolicyProvider caches the PermissionGrant's Permission's at
      grant time to guard against mutation, and looks these up from a
      ConcurrentMap, removing this map would allow the Permission's to
      be retrieved by direct reference instead.  To protect the
      PermissionGrant's from being implemented by anyone, I'll create a
      Permission the PermissionGrant's ProtectionDomain must have and
      check this at grant time.

DelegateCombinerSecurityManager (not in use on these tests) will hide 
some of the single thread performance cost (enable security debug, 
you'll see most permission checks are repeated), especially when there 
are many ProtectionDomain's in use on the stack context, since it caches 
successful results for each AccessControlContext (context).  It also 
contains an executor, used to divide large context's into smaller 
problems.   Small contexts (ProtectionDomain [] array length < 4) are 
not passed to the executor, since these are either privileged or have 
possibly at least two domains with AllPermission in most cases and are 
instead executed in the current thread. 

The executor has a policy to execute tasks in the calling thread when 
all executor threads are busy, so it's opportunistic task division, it 
is hoped that as load increases, the size of the permission check cache 
increases, reducing the load on the executor and policy, so this should 
be very scalable.

To avoid context permission cache map memory problems, the context keys 
are weak references, while the permissions softly referenced values in 
Sets.  If the jvm is low on memory, the softly referenced Permission's 
are removed, potentially increasing the load on the executor, so it 
should find a balance between cpu and memory consumption:

ConcurrentMap<AccessControlContext,Set<Permission>> checked.

The policy combines with the security manager to avoid multi thread 
deadlock when PermissionCollection blocks, by producing duplicate 
PermissionCollection's on demand for other threads to utilise, the 
policy (ConcurrentPermissions actually) keeps one copy in cache to avoid 
duplication when demand is low.

This setup should perform very well under contention, but until someone 
volunteers to test it on some big iron, we won't know how well.



 [java] -----------------------------------------
     [java] SUMMARY =================================
     [java] Test Passed: OK
     [java] -----------------------------------------
     [java] Test Passed: OK
     [java] -----------------------------------------
     [java] Test Passed: OK
     [java] -----------------------------------------
     [java] Test Passed: OK
     [java] -----------------------------------------
     [java] Test Passed: OK
     [java] -----------------------------------------
     [java] # of tests started   = 5
     [java] # of tests completed = 5
     [java] # of tests passed    = 5
     [java] # of tests failed    = 0
     [java] -----------------------------------------
     [java]    Date finished:
     [java]       Thu Dec 29 07:40:07 EST 2011
     [java]    Time elapsed:
     [java]       1528 seconds


Total time: 25 minutes 36 seconds

Brian Murphy wrote:
>> On 28 December 2011 12:02, Peter Firmstone <jini@zeus.net.au> wrote:
>>> This just seems to be about making sure events arrive in order.  I'm not
>>> even sure where to start with this.
> On Wed, Dec 28, 2011 at 7:33 AM, Dan Creswell <dan.creswell@gmail.com>
>  wrote:
>> I notice there are a whole bunch of timeouts/pause variables there.
>> I'd be tempted to increase each of those somewhat and see what
>> happens.
>> I'd be wondering if perhaps some things aren't arriving in time
>> because the timeouts are too tight. That might be because your
>> concurrent implementations whilst scalable are actually more costly in
>> terms of straight-line performance (not unusual as a tradeoff).
> I believe Dan is correct (as usual). Although the test does
> verify order of arrival, because the wrong number of events
> arrived in the allotted time, the test declares failure before it
> gets to the order verification step.
> So I agree with Dan that tuning the timeouts is probably a good
> place to start.
> I hope this helps,
> Brian

View raw message