cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Shadix <jacobsha...@gmail.com>
Subject Re: cassandra node stops streaming data during nodetool rebuild
Date Fri, 07 Apr 2017 10:47:08 GMT
What version are you running? Do you see any errors in the system.log
(SocketTimeout, for instance)?

And what values do you have for the following in cassandra.yaml:
- - stream_throughput_outbound_megabits_per_sec
- - compaction_throughput_mb_per_sec
- - streaming_socket_timeout_in_ms

-- Jacob Shadix

On Fri, Apr 7, 2017 at 6:00 AM, Roland Otta <Roland.Otta@willhaben.at>
wrote:

> hi,
>
> we are trying to setup a new datacenter and are initalizing the data
> with nodetool rebuild.
>
> after some hours it seems that the node stopped streaming (at least
> there is no more streaming traffic on the network interface).
>
> nodetool netstats shows that the streaming is still in progress
>
> Mode: NORMAL
> Bootstrap 6918dc90-1ad6-11e7-9f16-51230e2be4e9
> Rebuild 41606030-1ad9-11e7-9f16-51230e2be4e9
>     /192.168.0.26
>         Receiving 257 files, 145444246572 bytes total. Already received
> 1 files, 1744027 bytes total
>             bds/adcounter_total 76456/47310255 bytes(0%) received from
> idx:0/192.168.0.26
>             bds/upselling_event 1667571/1667571 bytes(100%) received
> from idx:0/192.168.0.26
>     /192.168.0.188
>     /192.168.0.27
>         Receiving 169 files, 79355302464 bytes total. Already received
> 1 files, 81585975 bytes total
>             bds/ad_event_history 81585975/81585975 bytes(100%) received
> from idx:0/192.168.0.27
>     /192.168.0.189
>         Receiving 140 files, 19673034809 bytes total. Already received
> 1 files, 5996604 bytes total
>             bds/adcounter_per_day 5956840/42259846 bytes(14%) received
> from idx:0/192.168.0.189
>             bds/user_event 39764/39764 bytes(100%) received from
> idx:0/192.168.0.189
> Read Repair Statistics:
> Attempted: 0
> Mismatch (Blocking): 0
> Mismatch (Background): 0
> Pool Name                    Active   Pending      Completed   Dropped
> Large messages                  n/a         2              3         0
> Small messages                  n/a         0       68632465         0
> Gossip messages                 n/a         0         217661         0
>
>
>
> it is in that state for approx 15 hours now
>
> does it make sense waiting for the streaming to finish or do i have to
> restart the node, discard data and restart the rebuild?
>

Mime
View raw message