Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 11 May 2017 15:13:04 +0000 (UTC)
From: =?utf-8?Q?Julius_=C5=BDaromskis_=28JIRA=29?= <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13063579.1492021414000.187626.1494515584314@Atlassian.JIRA>
In-Reply-To: <JIRA.13063579.1492021414000@Atlassian.JIRA>
References: <JIRA.13063579.1492021414000@Atlassian.JIRA> <JIRA.13063579.1492021414663@jira-lw-us.apache.org>
Subject: [jira] [Commented] (CASSANDRA-13441) Schema version changes for
 each upgraded node in a rolling upgrade, causing migration storms
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 11 May 2017 15:13:11 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-13441?page=3Dcom.atla=
ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=
=3D16006613#comment-16006613 ]=20

Julius =C5=BDaromskis commented on CASSANDRA-13441:
----------------------------------------------

Hi, any workaround for this issue? I've hit this after upgrading from 3.0.9=
 to 3.0.13 and doing sstableupgrade. Noticed weird disk write patterns and =
started seeing migration tasks bouncing around. I've only managed to update=
 first of the 3 nodes. Migrations tasks have stopped after I've rebooted fi=
rst node.

{noformat}
Cluster Information:
        Name: cloud.zaromskis.lt cluster
        Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
        Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
        Schema versions:
                600b7268-d42a-3b72-8706-093b6c8cfaff: [10.240.0.6]
                77a40699-8e9e-35aa-834e-68c32e40a45a: [10.240.0.7, 10.240.0=
.8]
{noformat}

{noformat}
Datacenter: dc1
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Status=3DUp/Down
|/ State=3DNormal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns (effective)  Host ID          =
                     Rack
UN  10.240.0.6  284.95 GB  256          63.4%             d0d83d9d-0dec-45c=
d-9ca9-93515fa131f3  rack1
UN  10.240.0.7  288.53 GB  256          64.1%             6d9709a0-0e10-46a=
1-9afa-d106b74ca9e0  rack1
UN  10.240.0.8  326.31 GB  256          72.5%             5c969700-8bd9-49a=
4-9772-1284439f8364  rack1
{noformat}

The schema version of first node would not propagate to other nodes. I'm af=
raid further upgrades might create new schema versions? I can't afford to l=
ose any data. Any advise?

> Schema version changes for each upgraded node in a rolling upgrade, causi=
ng migration storms
> -------------------------------------------------------------------------=
-------------------
>
>                 Key: CASSANDRA-13441
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1344=
1
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Schema
>            Reporter: Jeff Jirsa
>            Assignee: Jeff Jirsa
>             Fix For: 3.0.14, 3.11.0, 4.0
>
>
> In versions < 3.0, during a rolling upgrade (say 2.0 -> 2.1), the first n=
ode to upgrade to 2.1 would add the new tables, setting the new 2.1 version=
 ID, and subsequently upgraded hosts would settle on that version.
> When a 3.0 node upgrades and writes its own new-in-3.0 system tables, it'=
ll write the same tables that exist in the schema with brand new timestamps=
. As written, this will cause all nodes in the cluster to change schema (to=
 the version with the newest timestamp). On a sufficiently large cluster wi=
th a non-trivial schema, this could cause (literally) millions of migration=
 tasks to needlessly bounce across the cluster.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org