I am planning on setting up a Cassandra cluster on a small 16 node cluster (possibly 32 way).  Each machine has 8 cores 32 Gig of ram and 8 hds.  My first thought is to setup one of those hds for the commit log, 6 for data and leave one for the OS.  However I do have a concern about best utilizing my memory, should I run a larger heap?  Should I run several cassandra processes on the same box?

My concern about the larger heap is because GC's typically get slower.  And if I run several procs, does cassandra realize that it's the same box for replication purposes?

I do have other hd conf options, hardware RAID 0,1,or 5.

Just looking for some general configuration options as well as some real world successes with similarly sized hardware.  Thanks!

Aaron