river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Lookup Service Discovery using DNS?
Date Tue, 12 Jan 2010 21:00:01 GMT
Christopher Dolan wrote:
> I've seen v2.1 Reggie scaled up to 4000 registered services and 2000
> event listeners on a LAN, and I've seen Reggie working across WAN
> systems about half that size.  In the steady state in healthy systems,
> this works great.  A couple of problems I've seen:
>
>
>  1) when Reggie starts (or restarts), it gets hammered with new requests
> as services and clients find it via lookup locator polling, and chaos
> ensues.  I've seen Reggie go non-linear and hang when it exceeds N
> threads all waiting for RegistrarImpl.concurrentObj, where N is
> something like 1000.  In that regime, stack memory per thread matters a
> lot too.
>
>  2) the implementation of RegistrarProxy.lookup(ServiceTemplate,int)
> insists on unmarshalling all matches serially, which hangs for a long
> time if any services have bad codebase jars (denial of service). This
> weakest-link problem is exacerbated by WAN latencies.  I have an
> experimental patch that unmarshals in parallel, but it requires
> cooperation of both Reggie and clients (i.e. changes to both
> RegistrarProxy and ServiceDiscoveryManager).
>   
Chris, can you upload your experimental patch?

This looks like a problem that needs addressing.

Cheers,

Peter.

> The latter problem is solvable, but the former is hard.  Reggie is
> really sensitive to performance bottlenecks at startup.  Rewriting
> ReadersWriter to avoid using notifyAll() might help, perhaps borrowing
> CAS tricks from ReentrantReadWriteLock.
>
> Chris
>
>   


Mime
View raw message