incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From huyle <hu...@springpartners.com>
Subject Decommission a node causing high IO usage on 2 other nodes
Date Tue, 29 Nov 2011 22:30:26 GMT
Hi,

We recently upgraded a cluster to larger servers by bootstrapping the new
servers using token - 1 from old servers as the tokens for new servers.
After bootstrapping the new servers, old servers are shown owning 0.00%
workload ( they still really own a very small amount of data as I understand
it).   After decommissioning one of the old node, 2 new nodes next in the
token ring got IO wait from normal of 3% to 60%+.  Then we ran compaction on
the 2 new nodes that got high IO wait, IO is at 30%.  That 10 times the IO
wait we had before.  

Here is snippet of token ring for illustration purpose:

1.new
1.old
2.new
2.old
3.new
3.old
etc...

after decommission 1.old, 2.new and 3.new got very high IO wait.

What may be causing high IO wait on 2.new and 3.new?

Thanks!

Huy


--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Decommission-a-node-causing-high-IO-usage-on-2-other-nodes-tp7044866p7044866.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Mime
View raw message