cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Shuler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10291) Bootstrap hangs on adding new node
Date Fri, 11 Sep 2015 16:19:46 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741064#comment-14741064
] 

Michael Shuler commented on CASSANDRA-10291:
--------------------------------------------

Thanks for the extra info. This log shows C* 2.2.1 - are *all* nodes in the cluster running
2.2.1? I'm just trying to rule out the obvious.

> Bootstrap hangs on adding new node 
> -----------------------------------
>
>                 Key: CASSANDRA-10291
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10291
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Debian 7 64 bit
> HotSpot JDK 1.7.0_79
> Cassandra-2.2.1 via apt-get 
> 1x Intel Quad-Core Xeon E3-1230 / 16GB / 4x1TB SATA / 3x1TB RAID0 data drive 
>            Reporter: Ara Sadoyan
>         Attachments: nodetool.txt, system.log
>
>
> Adding new node in heavy loaded environment freeze bootstrap. No errors are reported
in log files.  Some of other other nodes throws "String didn't validate" error, but I;m not
sure that this is related. 
> After restarting node it start bootstrap again and hangs after some time . 
> nodetool netstats shows : 
> /data/XXX/XXXX/tmp-la-1184-big-Data.db 5126078789/18345924701   bytes(27%)  received
 from idx:0/192.168.220.16
> /data/XXX/XXXX/tmp-la-1233-big-Data.db 7213706459/18600941671   bytes(38%)  received
 from idx:0/192.168.220.22
> /data/XXX/XXXX/tmp-la-1599-big-Data.db 8492408759/17572043398   bytes(48%)  received
 from idx:0/192.168.220.12
> /data/XXX/XXXX/tmp-la-2066-big-Data.db 15773981555/18508127610  bytes(85%)  received
 from idx:0/192.168.220.18
> /data/XXX/XXXX/tmp-la-211-big-Data.db 8274231066/17172754085   bytes(48%)  received 
from idx:0/192.168.220.20
> but listing files on local FS shows "No such file or directory"
> This happens only if there is significant amount of data. I have 1.5 TB per node on 13
node cluster, we use STCS compaction strategy and flat network topology . 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message