ok, so I don't send writes to bootstrapping or decommissioned nodes, that's cool, but what about the inconsistent ring view after nodetool move, isn't this strange?
After the move, the moved node has the correct view of the ring but all other nodes have the old view. I waited a few minutes after the log said that Bootstrap/move completed! Now serving reads but this didn't help, view was still inconsistent. Only restarting the moved node helped other nodes realize the change.

On Wed, Aug 4, 2010 at 3:24 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
Don't point clients at nodes that aren't part of the ring.  Cassandra
rejecting requests when you do is a "feature."

On Wed, Aug 4, 2010 at 6:52 AM, Ran Tavory <rantav@gmail.com> wrote:
> Is this a known issue?
> Running 0.6.2 I moved a node to different token and eventually saw errors in
> the log.
>
> ERROR [ROW-READ-STAGE:116804] 2010-08-04 06:34:29,699
> DebuggableThreadPoolExecutor.java (line 101) Error in ThreadPoolExecutor
> java.lang.RuntimeException: Cannot service reads while bootstrapping!
>         at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:66)
>         at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> ERROR [ROW-READ-STAGE:116805] 2010-08-04 06:34:29,700 CassandraDaemon.java
> (line 82) Fatal exception in thread Thread[ROW-READ-STAGE:116805,5,main]
> java.lang.RuntimeException: Cannot service reads while bootstrapping!
>         at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:66)
>         at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> ... many more of those and then...
>  INFO [MESSAGE-DESERIALIZER-POOL:1] 2010-08-04 06:34:29,709
> StorageService.java (line 181) Bootstrap/move completed! Now serving reads.
>
> The move ended up "ok" but during the operation the log was filled with
> those errors and at the end of it the ring state was inconsistent.
> If I ask the moved node where it is in the ring it tells me something but
> other nodes tell something else...
> (ob1124)(cassandra@cass24:apache-cassandra-0.6.2)$ nodetool -h
> 192.168.254.58 -p 9004 ring
> Address       Status     Load          Range
>      Ring
>
> 170141183460469231731687303715884105727
> 192.168.252.88Up         5.7 GB
>  14131484407726020523932116250949797205     |<--|
> 192.168.252.124Up         2.44 GB
> 56713727820156410577229101238628035242     |   ^
> 192.168.254.58Up         8.13 GB
> 113427455640312821154458202477256070484    v   |
> 192.168.254.57Up         6.52 GB
> 113427455640312821154458202477256070485    |   ^
> 192.168.252.125Up         6.52 GB
> 141784319550391026443072753096570088105    v   |
> 192.168.254.59Up         1.63 GB
> 170141183460469231731687303715884105727    |-->|
> (ob1124)(cassandra@cass24:apache-cassandra-0.6.2)$ nodetool
> -h 192.168.252.124 -p 9004 ring
> Address       Status     Load          Range
>      Ring
>
> 170141183460469231731687303715884105727
> 192.168.252.88Up         5.7 GB
>  14131484407726020523932116250949797205     |<--|
> 192.168.252.124Up         2.46 GB
> 56713727820156410577229101238628035242     |   ^
> 192.168.254.57Up         6.52 GB
> 113427455640312821154458202477256070485    v   |
> 192.168.252.125Up         6.52 GB
> 141784319550391026443072753096570088105    |   ^
> 192.168.254.58Up         1.63 GB
> 141784319550391026443072753096570088106    v   |
> 192.168.254.59Up         1.63 GB
> 170141183460469231731687303715884105727    |-->|
> Restarting the moved node fixes the ring view by other hosts.
>
>



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com