I don't see any errors in neither the receiver nor sender. The cluster itself looks healthy (e.g. no random networks disconnections or anything like that as described in 1019).

I'll restart the moved node, hope it'll continue its more afterwards but it does look like a bug. Is there something I can do to help debug it?
I'll attach a thread dump here just in case before I restart it



On Wed, May 26, 2010 at 9:54 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
Are there any exceptions in the log like the one in
https://issues.apache.org/jira/browse/CASSANDRA-1019 ?

If so you'll need to restart the moving node and try again.

On Wed, May 26, 2010 at 3:54 AM, Ran Tavory <rantav@gmail.com> wrote:
> I ran nodetool move on one of the nodes and it seems stuck for a few hours
> now.
> I've been able to run it successfully in the past, but this time it looks
> stuck.
> Streams shows as if there's work in progress, but the same files have been
> at the same position for a few hours.
> I've also checked the compaction manager through jmx and it's not compacting
> anything.
> Anything I can do to continue the move operation? Is this a bug?
> thanks
>
> $ nodetool -h 192.168.254.58 -p 9004 streams
> Mode: Normal
>  Nothing streaming to /192.168.252.124
> Streaming to: /192.168.254.57
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvImpressions-170-Filter.db
> 0/45587965
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvImpressions-170-Data.db
> 0/18369406636
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvRatings-142-Index.db
> 0/6163411
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvRatings-142-Filter.db
> 0/85645
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvRatings-142-Data.db
> 0/37985032
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvAds-108-Index.db
> 0/77307418
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvAds-108-Filter.db
> 0/1436005
>    /outbrain/cassandra/data/outbrain_kvdb/stream/KvAds-108-Data.db
> 0/704736670
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-40-Index.db 0/230280
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-40-Filter.db 0/20605
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-40-Data.db
> 0/202808000
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-41-Index.db 0/23912
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-41-Filter.db 0/20605
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-41-Data.db 0/21434973
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-42-Index.db 0/965028
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-42-Filter.db 0/20605
>    /outbrain/cassandra/data/Keyspace1/stream/Standard1-42-Data.db
> 0/865380205
> Not receiving any streams.
> $ nodetool -h 192.168.254.57 -p 9004 streams
> Mode: Normal
>  Nothing streaming to /192.168.254.58
> Streaming from: /192.168.254.58
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvImpressions-tmp-1081-Data.db
> 0/18369406636
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvRatings-tmp-167-Index.db 0/6163411
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvRatings-tmp-167-Filter.db 0/85645
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvRatings-tmp-167-Data.db 0/37985032
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvAds-tmp-123-Index.db 0/77307418
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvAds-tmp-123-Filter.db 0/1436005
>    outbrain_kvdb:
> /outbrain/cassandra/data/outbrain_kvdb/KvAds-tmp-123-Data.db 0/704736670
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-21-Index.db
> 0/230280
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-21-Filter.db
> 0/20605
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-21-Data.db
> 0/202808000
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-22-Index.db
> 0/23912
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-22-Filter.db
> 0/20605
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-22-Data.db
> 0/21434973
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-23-Index.db
> 0/965028
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-23-Filter.db
> 0/20605
>    Keyspace1: /outbrain/cassandra/data/Keyspace1/Standard1-tmp-23-Data.db
> 0/865380205
>
>
>



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com