incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Ciuloaica <gciuloa...@gmail.com>
Subject Re: Adding new nodes to an existing cluster
Date Fri, 12 Apr 2013 14:53:23 GMT
Thanks Marcus,

So, at this moment I have to wait until release of version 1.2.5 to be 
able to extend the cluster ... I haven't seen mention any workaround ...

Thanks,
Gabi
On 4/12/13 11:11 AM, Marcus Eriksson wrote:
> looks very similar to this:
> https://issues.apache.org/jira/browse/CASSANDRA-5418
>
> /Marcus
>
>
> On Fri, Apr 12, 2013 at 9:12 AM, Gabriel Ciuloaica 
> <gciuloaica@gmail.com <mailto:gciuloaica@gmail.com>> wrote:
>
>     Hi,
>
>     From yesterday, I'm trying to add a new node to an existing 3
>     nodes Cassandra cluster, running version 1.2.3.  Today I have
>     started clean, removed the node that I have tried to add yesterday
>     and try to add another node. Verified both cassandra.yaml and
>     cassandra-topology.properties are identical on all nodes (I'm
>     using NetworkTopologyStrategy and PropertyFile snitch).
>     I see the nodetool netstats on the new node shows that it is
>     streaming from the other nodes, but other than increasing number
>     on Responses Pool, there is no change in the report.  Logs are set
>     to Debug and most of new lines are "Applying mutations".
>
>     Below is the only ERROR that I can find in both cassandra.log and
>     system.log.  So, it seems that streaming is working but not
>     completing... For the sstable that I get the error, the percentage
>     is not increasing in netstats, not for any other sstables. Now it
>     is running for few hours and no progress. Any clue ?
>     ERROR 06:09:03,934 Exception in thread Thread[Thread-25,5,main]
>     java.lang.AssertionError: incorrect row data size 114969 written
>     to
>     /data/woow_backend/offline/woow_backend-offline-tmp-ib-2-Data.db;
>     correct is 115006
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:285)
>             at
>     org.apache.cassandra.streaming.IncomingStreamReader.streamIn(IncomingStreamReader.java:179)
>             at
>     org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:122)
>             at org.apache.cassandra.net
>     <http://org.apache.cassandra.net>.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
>             at org.apache.cassandra.net
>     <http://org.apache.cassandra.net>.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
>             at org.apache.cassandra.net
>     <http://org.apache.cassandra.net>.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
>
>
>
>     nodetool netstats:
>
>     Mode: JOINING
>     Not sending any streams.
>     Streaming from: /10.11.1.108 <http://10.11.1.108>
>        woow_backend:
>     /data/woow_backend/user_rooms/woow_backend-user_rooms-ib-1-Data.db
>     sections=1 progress=0/566 - 0%
>        woow_backend:
>     /data/woow_backend/gc_archive/woow_backend-gc_archive-ib-2-Data.db
>     sections=1 progress=0/3478 - 0%
>        woow_backend:
>     /data/woow_backend/msg_archive/woow_backend-msg_archive-ib-258-Data.db
>     sections=1 progress=0/313 - 0%
>        woow_backend:
>     /data/woow_backend/roster_groups/woow_backend-roster_groups-ib-23-Data.db
>     sections=1 progress=0/120 - 0%
>        woow_backend:
>     /data/woow_backend/roster_versions/woow_backend-roster_versions-ib-34-Data.db
>     sections=2 progress=0/195 - 0%
>        woow_backend:
>     /data/woow_backend/rosters/woow_backend-rosters-ib-27-Data.db
>     sections=93 progress=0/881196 - 0%
>        woow_backend:
>     /data/woow_backend/msg_archive/woow_backend-msg_archive-ib-257-Data.db
>     sections=104 progress=0/5816889 - 0%
>        woow_backend:
>     /data/woow_backend/avatars/woow_backend-avatars-ib-68-Data.db
>     sections=71 progress=0/5841975 - 0%
>        woow_backend:
>     /data/woow_backend/rosters/woow_backend-rosters-ib-28-Data.db
>     sections=2 progress=0/234 - 0%
>        woow_backend:
>     /data/woow_backend/msg_archive/woow_backend-msg_archive-ib-259-Data.db
>     sections=3 progress=0/6910 - 0%
>        woow_backend:
>     /data/woow_backend/offline/woow_backend-offline-ib-129-Data.db
>     sections=77 progress=22499/462727 - 4%
>        woow_backend:
>     /data/woow_backend/rooms/woow_backend-rooms-ib-2-Data.db
>     sections=1 progress=0/308 - 0%
>        woow_backend:
>     /data/woow_backend/roster_versions/woow_backend-roster_versions-ib-33-Data.db
>     sections=93 progress=0/34246 - 0%
>        woow_backend:
>     /data/woow_backend/rooms/woow_backend-rooms-ib-1-Data.db
>     sections=1 progress=0/310 - 0%
>     Pool Name                    Active   Pending Completed
>     Commands                        n/a         0             15
>     Responses                       n/a         0          44808
>
>     Thanks,
>     Gabi
>
>


Mime
View raw message