Well... Strange. We have such problem with 6 users, but there's only ONE tombstone (created 8 days ago, so it's not gcable yet) in all the SSTables on 2:1 node - checked using sstable2json.
Moreover, this tombstone DOES NOT belong to the row key I'm using for tests, because this user was NOT ever removed / replaced.
So now I have no bloody idea how C* can see a tombstone for this key when running a query. Maybe it's a index problem then?
Yes, maybe there are two issues here: repair not running and maybe really some index-thing.
Maybe you can try a CL=ONE with cassandra-cli? So that we can see how it works without index.
it tells me about "Key cache hit for sstable" for SSTables 23 & 24, but I don't have such SSTables for Users CF. However, I have SSTables like this for index.
I think the Index-SSTables and the data SSTables are compacted separately and the numbers can differ from the data, even though they are flushed together. So the numbers can differ. (anybody feel free to correct me on this)