cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mck SembWever (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-4417) invalid counter shard detected
Date Wed, 07 Nov 2012 10:21:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492245#comment-13492245
] 

Mck SembWever edited comment on CASSANDRA-4417 at 11/7/12 10:20 AM:
--------------------------------------------------------------------

Sylvain, here's log from one node. For most of the log we were running 1.0.8. And then at
line 2883399 we upgraded (and this was the first node to upgrade) to 1.1.6.

The error msg comes every few seconds.
Our counters are sub-columns inside supercolumns.
We completed the upgrade on all nodes. Then restarted again (because jna was missing).

We are now running upgradesstables but that's not in this logfile. The error msgs still appear.

An operational problem we've had recently is that we had one node down for ~one month (faulty
raid controller) and when we finally brought the node back into the cluster nightly repairs
would never finish. In the end we just disabled nightly repairs (we don't have tombstones)
with the plan that an upgrade and upgradesstables would bring us back to a state where repairs
would work again. I have no idea if this can be related. 
                
      was (Author: michaelsembwever):
    Sylvain, here's log from one node. For most of the log we were running 1.0.8. And then
at line 2883399 we upgraded (and this was the first node to upgrade) to 1.1.6.

The error msg comes every few seconds.
Our counters are sub-columns inside supercolumns.
We completed the upgrade on all nodes. Then restarted again (because jna was missing).

We are now running upgradesstables but that's not in this logfile. The error msgs still appear.

An operational problem we're had recently is that we had one node down for ~one month (faulty
raid controller) and when we finally brought the node back into the cluster nightly repairs
would never finish. In the end we just disabled nightly repairs (we don't have tombstones)
with the plan that an upgrade and upgradesstables would bring us back to a state where repairs
would work again. I have no idea if this can be related. 
                  
> invalid counter shard detected 
> -------------------------------
>
>                 Key: CASSANDRA-4417
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4417
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.1
>         Environment: Amazon Linux
>            Reporter: Senthilvel Rangaswamy
>         Attachments: cassandra-mck.log.bz2, err.txt
>
>
> Seeing errors like these:
> 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; (17bfd850-ac52-11e1-0000-6ecd0b5b61e7,
1, 13) and (17bfd850-ac52-11e1-0000-6ecd0b5b61e7, 1, 1) differ only in count; will pick highest
to self-heal; this indicates a bug or corruption generated a bad counter shard
> What does it mean ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message