am guessing a lot here, but I would check if auto_bootstrap is enabled. It is by default.
When a new node joins reads are not directed to it until it is marked as "UP" (writes are sent to it as it is joining). So reads should continue to go to the original UP node.
Sounds like it's all running now. If you get see it again can you provide some detailed steps such as the type of query and the CL level.
On 2/03/2012, at 7:35 AM, Casey Deccio wrote:
On Thu, Mar 1, 2012 at 9:33 AM, aaron morton <firstname.lastname@example.org> wrote:
What RF were you using and had you been running repair regularly ?
RF 1 *sigh*. Waiting until I have more/better resources to use RF > 1. Hopefully soon.
In the mean time... Oddly (to me), when I removed the most recently added node, all my rows re-appeared, but were only up-to-date as of a 10 days ago (a few days before I added the node). None of the supercolumns since then show up. But when I look at the sstable files on the different nodes, I see large files with timestamps in between that date and today's, which makes me think the data is still there. Also, if I re-add the troublesome new node (not having run cleanup), all rows are again inaccessible until I again decommission it.