I think I finally found what. It's implementation of Java NIOon Windows (JVM 18.104.22.168, 64b on Windows 2003)
The very same code, same network but CentOS linux gives almost 4x performance. (in Cassandra@linux -> Cassandra@Windows setup)
I don't have another linux box to test (Cassandra@linux -> Cassandra@linux) performance, but expect it to be even better.
A lesson learnt: don't use windows.
Here at Viigo we also learnt the hard way that async IO is also broken in .NET (C#). Now I start to wonder if there is some fundamental flaw in async IO on windows...
On Mon, Oct 5, 2009 at 2:17 PM, Igor Katkov <firstname.lastname@example.org> wrote:this is going up basically linearly with amount of (data x clients),
> measured via JMX console i.e. does not include client-cassandra-client
> 20 client threads 176975b value StorageProxy.WriteLatency ~660ms
> 10 client threads 176975b value StorageProxy.WriteLatency ~350ms
> 05 client threads 176975b value StorageProxy.WriteLatency ~156ms
so clearly something is getting saturated.