cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Love (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11983) Migration task failed to complete
Date Sun, 12 Jun 2016 18:20:21 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326568#comment-15326568
] 

Chris Love commented on CASSANDRA-11983:
----------------------------------------

I need a pattern where the same docker is used for expanding an existing cluster as well as
creating an empty cluster. To that point launching with autobootstrap: false is not feasible.
Kubernetes provides the capability to scale up, but it uses the same docker to do that with,
so the docker has to support the case where the node bootstraps. Besides 2.2.x is working
like a champ. Starting more than one node a second, and I took it up to 300 nodes. There has
been a change introduced into a version of 3 that is causing this behavior.

Should I turn on specific logging? What exactly is the node doing during the Migration Task?

I have backed out the code, but have not tested. I do understand that the code was a fix for
another issue, but it appears that the node gets stuck there. I may be wrong, but having that
latch timing out in a loop, seems to be a challenge where the node not fail gracefully.  It
just waits for something that will not happen ;(

> Migration task failed to complete
> ---------------------------------
>
>                 Key: CASSANDRA-11983
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11983
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Lifecycle
>         Environment: Docker / Kubernetes running
> Linux cassandra-21 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-1 (2016-03-06) x86_64 GNU/Linux
> openjdk version "1.8.0_91"
> OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-1~bpo8+1-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassnadra 3.5 installed from 
> deb-src http://www.apache.org/dist/cassandra/debian 35x main
>            Reporter: Chris Love
>         Attachments: cass.log
>
>
> When nodes are boostrapping I am getting mulitple errors: "Migration task failed to complete",
from MigrationManager.java
> The errors increase as more nodes are added to the ring, as I am creating a ring of 1k
nodes.
> Cassandra yaml i here https://github.com/k8s-for-greeks/gpmr/blob/3d50ff91a139b9c4a7a26eda0fb4dcf9a008fbed/pet-race-devops/docker/cassandra-debian/files/cassandra.yaml



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message