cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Malone <>
Subject Re: 10G Ethernet / Infiniband
Date Wed, 27 Oct 2010 17:00:04 GMT
If you're talking about throughput (bandwidth) you would need at least 4
full speed SATA2 disks in a RAID to even have a chance of saturating 10Gbps
ethernet with data off disk. Rotational drives are slower, but cheaper. As
far as which are "better" in the current environment: not gunna touch that
with a 10 foot pole.

Cassandra caches heavily though. It's a NUMA-ish architecture. Lots of stuff
comes out of memory. And it does a lot of message passing. So I would
anticipate some performance improvements. Not sure whether it would be
better to invest money in 10Gbps networking vs. faster cpus, more disks, or
more nodes though.

Amazon EC2 recently started letting you spin up HPC instances that are
supposed to be rack-local with one another with full 10Gbps ethernet
networking. We've been talking about testing Cassandra out in that
environment at SimpleGeo to see if it's faster, since we're already on EC2.
If/when we do I can report back. But I'm not sure what our timeline is on
that. It may be faster for you to do your own tests there. Not the cleanest
environment, but it's better than rumors and guesses.


On Wed, Oct 27, 2010 at 6:29 AM, Juho Mäkinen <>wrote:

> The disks in cassandra node will most probably be your bottleneck. I'd
> suggest (haven't tried, this is just based on my intuition) to invest
> in SSD disks first and only after that think about going 10Gbps.
>  - Garo
> On Tue, Oct 26, 2010 at 10:11 PM, Wayne <> wrote:
> > Is anyone out there using 10G Ethernet or Infiniband with Cassandra in a
> > cluster where there is a noticeable increase in performance or reduced
> > latency to warrant the cost of the faster network connections?
> >
> > Any experience or even rationale would help. We are considering
> Infiniband
> > but are on the fence as to how much it would really improve performance
> and
> > reduce latency (along with how stable CentOS 5.5 drivers would be).
> >
> > Thanks in advance for any information/feedback.
> >
> >
> >

View raw message