incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jared Laprise <>
Date Fri, 25 Mar 2011 03:44:59 GMT
Thanks for the responses. I got everything working again, and have some ideas on why but am
not completely sure.

How I got it working again was simply bring the second node back online. I was under the assumption
that all data is replicated between nodes (eventually). Am I incorrect? It would seem that
each node stores different data and delegates the read request to whichever node holds the
data. Although I've spent a lot of time with Cassandra in a single node environment I think
I may be lacking a bit of understanding on how Cassandra behaves in a clustered environment.

-----Original Message-----
From: Jonathan Ellis [] 
Sent: Thursday, March 24, 2011 8:22 PM
Cc: aaron morton

Right, Cassandra doesn't keep old versions around so to see an old version you have to have
uncompacted data and whack the new data -- either by blowing away sstables or not replaying
the commitlog.

Snapshots flush before creating their hard links, which rules out any commitlog problems.

If you ran out of disk space you wouldn't get past the commitlog append, so you'd never get
new data in at all after that.

Sounds like an environmental problem, not Cassandra specific.

On Thu, Mar 24, 2011 at 9:10 PM, aaron morton <> wrote:
> Was there anything in the server logs during startup ?
> I've not heard of this happening before and it's hard think of how / 
> why cassandra could revert it's data. Other than something external 
> playing with the files on disk Aaron On 25 Mar 2011, at 13:49, Jared 
> Laprise wrote:
> Hello all, I'm running 2 Cassandra 6.5 nodes and I brought down the 
> secondary node and restarted the primary node. After Cassandra came 
> back up all data has been reverted to several months ago.
> I could really use some incite here, this is a production website and 
> I need to act quickly. I have a cron job that takes a snapshot every 
> night, but even with that I tried to restore a snapshot on my local 
> development environment and it was also missing a ton of data.
> Any help will be so appreciated.

Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support

View raw message