And three days later, AE stages are still running full-bore. So I conclude this is not a very good approach.
I wonder what will happen when I lose a disk (which is essentially the same as what I did -- rm the data directory). What happens if I lose a disk while the AE stages are running? Since my RF is 3, I assume that I have data loss when three disks are gone.
Not very happy. I'm going to blow away what I have, do another reload, then try dropping a disk again, just to confirm the results... I can't really believe this is how it should happen.
Story continued, in hopes this experience is useful to someone...
I shut down the node, removed the huge file, restarted the node, and told everybody to repair. Two days later, AE stages are still running.
On Thu, Jun 3, 2010 at 2:21 AM, Jonathan Ellis <email@example.com> wrote:this is why JBOD configuration is contraindicated for cassandra.
On Tue, Jun 1, 2010 at 1:08 PM, Ian Soboroff <firstname.lastname@example.org> wrote:
--> My nodes have 5 disks and are using them separately as data disks. The
> usage on the disks is not uniform, and one is nearly full. Is there some
> way to manually balance the files across the disks? Pretty much anything
> done via nodetool incurs an anticompaction with obviously fails. system/ is
> not the problem, it's in my data's keyspace.
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support