The tokens were different than the production cluster and after closer inspection a lot of data wasn't queryable (as excpected I suppose). I set the tokens and everything seems ok now.
>>>>> "Eric" == Eric Czech <email@example.com> writes:Eric> Yea that's not a mapping I'd like to maintain either -- as an
Eric> experiment, I copied production sstables to the analysis
Eric> cluster and ran brisk/cassandra without specifying an initial
Eric> token (after deleting the LocationInfo* files and renaming the
Based on my understanding this will allow everything to start up, yes.
Eric> As far as I can tell, everything is running normally but I'm
Eric> not sure how the cluster chose tokens for the nodes given that
Eric> I didn't specify them after just dropping the raw sstables
Eric> in. I can still read data as usual from the column families
Eric> that were copied but I'm not sure how not specifying the
Eric> tokens affects everything.
Did you check the ring to see what tokens you got for the analysis
cluster? I would be surprised if you got the same ring configuration as
Eric> Is some of my data just unreachable now because the tokens
Eric> weren't manually defined?
I suspect your data is messed up. But the best way to determine it would
be to examine the ring (use nodetool) - if it is the same as your
production cluster you are good to go.
Also, did you set your (non seed) nodes in the analysis cluster to auto
bootstrap or not? That impacts what happens.
Eric> This doesn't appear to be the case but is this something you
Eric> have tried too or do you understand the storage / topology
Eric> logic well enough to know that this isn't a viable strategy?
No and No. I have been reading the code. Line 497 of
org.apache.cassandra.service.StorageService.java on trunk is a good
place to start since what happens depends somewhat on your specific
cassandra.yaml settings (specifically auto bootstrap).
I would be betting you are getting random tokens (look for "Generated
random token..." in your log). Don't trust me, read the code. I have all
of two weeks of experience with this stuff (and it's not quite my day
job to be doing it either :-)
Bottom line: I think you need to fix the seeds for your use case.