Here is an example of the increase for some counter (counting events per
hour)
time (UTC) 0 1 2 3 4 5 6 7 8 9 10 11 12 13
Good value 88 44 26 35 26 86 187 251 455 389 473 367 453 373
C* counter 149 82 45 68 38 146 329 414 746 566 473 377 453 373
I finished my Cassandra 1.1.6 upgrades at 9:30 UTC.
I found wrong values since the day before at 20:00 UTC (counters from hours
before are good)
Here are the logs from the output:
Server 1: http://pastebin.com/WyCm6Ef5 (This one is from the same server as
the first bash history on my first mail)
Server 2: http://pastebin.com/gBe2KL2b (This one is from the same server
as the second bash history on my first mail)
Alain
2012/11/15 aaron morton <aaron@thelastpickle.com>
> Can you provide an example of the increase ?
>
> Can you provide the log from startup ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 16/11/2012, at 3:21 AM, Alain RODRIGUEZ <arodrime@gmail.com> wrote:
>
> We had an issue with counters over-counting even using the nodetool drain
> command before upgrading...
>
> Here is my bash history
>
> 69 cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak
> 70 cp /etc/cassandra/cassandra-env.sh
> /etc/cassandra/cassandra-env.sh.bak
> 71 sudo apt-get install cassandra
> 72 nodetool disablethrift
> 73 nodetool drain
> 74 service cassandra stop
> 75 cat /etc/cassandra/cassandra-env.sh
> /etc/cassandra/cassandra-env.sh.bak
> 76 vim /etc/cassandra/cassandra-env.sh
> 77 cat /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak
> 78 vim /etc/cassandra/cassandra.yaml
> 79 service cassandra start
>
> So I think I followed these steps
> http://www.datastax.com/docs/1.1/install/upgrading#upgrade-steps
>
> I merged my conf files with an external tool so consider I merged my conf
> files on steps 76 and 78.
>
> I saw that the "sudo apt-get install cassandra" stop the server and
> restart it automatically. So it updated without draining and restart before
> I had the time to reconfigure the conf files. Is this "normal" ? Is there a
> way to avoid it ?
>
> So for the second node I decided to try to stop C*before the upgrade.
>
> 125 cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak
> 126 cp /etc/cassandra/cassandra-env.sh
> /etc/cassandra/cassandra-env.sh.bak
> 127 nodetool disablegossip
> 128 nodetool disablethrift
> 129 nodetool drain
> 130 service cassandra stop
> 131 sudo apt-get install cassandra
>
> //131 : This restarted cassandra
>
> 132 nodetool disablethrift
> 133 nodetool disablegossip
> 134 nodetool drain
> 135 service cassandra stop
> 136 cat /etc/cassandra/cassandra-env.sh
> /etc/cassandra/cassandra-env.sh.bak
> 137 cim /etc/cassandra/cassandra-env.sh
> 138 vim /etc/cassandra/cassandra-env.sh
> 139 cat /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.bak
> 140 vim /etc/cassandra/cassandra.yaml
> 141 service cassandra start
>
> After both of these updates I saw my current counters increase without any
> reason.
>
> Did I do anything wrong ?
>
> Alain
>
>
>
|