If your replication factor is set to one, your cluster is obviously in a bad state following any node failure.  At best, I think it would make sense that about a third of your operations fail, but I'm not sure why all of them would.  I don't know if Hector just refuses to work with a compromised cluster, etc.

I guess I'm wondering why your replication factor is set to 1

Caleb Rackliffe | Software Developer
M 949.981.0159 | caleb@steelhouse.com

From: Tamar Fraenkel <tamar@tok-media.com>
Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Sun, 18 Mar 2012 03:15:53 -0400
To: "cassandra-user@incubator.apache.org" <cassandra-user@incubator.apache.org>
Subject: consistency level question

Hi!
I have a 3 node cassandra cluster.
I use Hector API.

I give hecotr one of the node's IP address
I call setAutoDiscoverHosts(true) and setRunAutoDiscoveryAtStartup(true).

The describe on one node returns:

Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:1]

The odd thing is that when I take one of the nodes down, expecting all to continue running smoothly, I get exceptions of the format seen bellow, and no read or write succeeds. When I bring the node back up, exceptions stop and read and write resumes.

Any idea or explanation why this is the case?
Thanks!


me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level.
        at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:66)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
        at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
        at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289)
        at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53)
        at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49)
        at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
        at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
        at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48)
        at me.prettyprint.cassandra.service.ColumnSliceIterator.hasNext(ColumnSliceIterator.java:60)
        at 


Tamar Fraenkel 
Senior Software Engineer, TOK Media 

Inline image 1