What is the consistency level you are using ?
btw, the nodes are a tad out of balance was that deliberate ?AaronOn 10 Apr 2011, at 08:44, Ed Anuff wrote:Sounds like the problem might be on the hector side. Lots of hector
users on this list, but usually not a bad idea to ask on
The jetty servers stopping responding is a bit vague, somewhere in
your logs is an error message that should shed some light on where
things are going awry. If you can find the exception that's being
thrown in hector and post that, it'd make it much easier to help you
On Sat, Apr 9, 2011 at 12:11 PM, Vram Kouramajian
<email@example.com> wrote:The hector clients are used as part of our jetty servers. And, thejetty servers stop responding when one of the Cassandra nodes go down.VramOn Sat, Apr 9, 2011 at 11:54 AM, Joe Stump <firstname.lastname@example.org> wrote:Did the Cassandra cluster go down or did you start getting failures from the client when it routed queries to the downed node? The key in the client is to keep working around the ring if the initial node is down.
--JoeOn Apr 9, 2011, at 12:52 PM, Vram Kouramajian wrote:We have a 5 Cassandra nodes with the following configuration:Casandra Version: 0.6.11Number of Nodes: 5Replication Factor: 3Client: Hector 0.6.0-14Write Consistency Level: QuorumRead Consistency Level: QuorumRing Topology:Owns Range Ring
132756707369141912386052673276321963528192.168.89.153Up 4.15 GB 33.87%20237398133070283622632741498697119875 |<--|192.168.89.155Up 5.17 GB 18.29%51358066040236348437506517944084891398 | ^
192.168.89.154Up 7.41 GB 33.97%109158969152851862753910401160326064203 v |192.168.89.152Up 5.07 GB 6.34%
119944993359936402983569623214763193674 | ^192.168.89.151Up 4.22 GB 7.53%132756707369141912386052673276321963528 |-->|
We believe that our setup should survive the crash of one of theCassandra nodes. But, we had few crashes and the system stopped
functioning until we brought back the Cassandra nodes.Any clues?Vram