We're having a problem with our cassandra cluster and are at a loss as to the cause.
We have what appear to be columns that disappear for a little while, then reappear. The rest of the row is returned normally during this time. This is, of course, very disturbing, and is wreaking havoc with our application.
A bit more info about what's happening:
We are repeatedly executing the same query against our cluster. Every so often, one of the columns will disappear from the row and will remain gone for some time. Then after continually executing the same query, the column will come back. The queries are being executed against a 3 node cluster, with a replication factor of 3, and all reads and writes are done with a quorum consistency level.
We upgraded from cassandra 1.1.12 to 1.2.6 last week, but only started seeing issues this morning.
Has anyone had a problem like this before, or have any idea what might be causing it?