Sometimes "resetlocalschema" (happened to me, didn't check why) will not work, and you need to stop the offending nodes and bring them back one by one.. That solved the issue.

In the cases I've seen this happened the clusters in question where either:

a) A couple of nodes down (cloud provider caused a partial outage) with an application that would create/change tables every now and then
b) Massive GC pauses around the cluster with a schema change being tried over and over.

CQLSH stopped working, your nodes died? GC Pauses? Operator mistake?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
 
Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: linkedin.com/in/carlosjuzarterolo
Mobile: +351 918 918 100

On Sat, May 6, 2017 at 1:07 PM, Nitan Kainth <nitan@bamlabs.com> wrote:
Thank you Jeff.

James,

We started getting insufficient replica errors. Cqlsh stopped working for two nodes .

Sent from my iPhone

> On May 5, 2017, at 7:13 PM, James Rothering <jrothering@codojo.me> wrote:
>
> I've heard about this ... how did the problem present itself?
>
> Sent from my iPhone
>
>> On May 5, 2017, at 3:17 PM, Jeff Jirsa <jjirsa@apache.org> wrote:
>>
>>
>>
>>> On 2017-05-05 11:00 (-0700), Nitan Kainth <nitan@bamlabs.com> wrote:
>>> Hi Experts,
>>>
>>> We found schema version mismatch in our cluster. We fixed it by bouncing C* on nodes where version was mismatched. Can someone suggest, what are the possible reasons for this? We are trying to figure out the root cause.
>>>
>>
>> Do all of your versions match? You didn't accidentally upgrade half the cluster did you?
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org



--