Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8A513200D57 for ; Mon, 11 Dec 2017 17:25:10 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 88FCC160C13; Mon, 11 Dec 2017 16:25:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CE4CE160C10 for ; Mon, 11 Dec 2017 17:25:09 +0100 (CET) Received: (qmail 83538 invoked by uid 500); 11 Dec 2017 16:25:09 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 83529 invoked by uid 99); 11 Dec 2017 16:25:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Dec 2017 16:25:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8A7861A0907 for ; Mon, 11 Dec 2017 16:25:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Qoyyo3mloWgi for ; Mon, 11 Dec 2017 16:25:07 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 686125FB9E for ; Mon, 11 Dec 2017 16:25:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 72E67E25A0 for ; Mon, 11 Dec 2017 16:25:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 05515212FF for ; Mon, 11 Dec 2017 16:25:05 +0000 (UTC) Date: Mon, 11 Dec 2017 16:25:05 +0000 (UTC) From: "Anton Vinogradov (JIRA)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (IGNITE-7165) Re-balancing is cancelled if client node joins MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 11 Dec 2017 16:25:10 -0000 [ https://issues.apache.org/jira/browse/IGNITE-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-7165: ------------------------------------- Fix Version/s: 2.4 > Re-balancing is cancelled if client node joins > ---------------------------------------------- > > Key: IGNITE-7165 > URL: https://issues.apache.org/jira/browse/IGNITE-7165 > Project: Ignite > Issue Type: Bug > Reporter: Mikhail Cherkasov > Assignee: Anton Vinogradov > Priority: Critical > Fix For: 2.4 > > > Re-balancing is canceled if client node joins. Re-balancing can take hours and each time when client node joins it starts again: > [15:10:05,700][INFO][disco-event-worker-#61%statement_grid%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=979cf868-1c37-424a-9ad1-12db501f32ef, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 172.31.16.213], sockAddrs=[/0:0:0:0:0:0:0:1:0, /127.0.0.1:0, /172.31.16.213:0], discPort=0, order=36, intOrder=24, lastExchangeTime=1512907805688, loc=false, ver=2.3.1#20171129-sha1:4b1ec0fe, isClient=true] > [15:10:05,701][INFO][disco-event-worker-#61%statement_grid%][GridDiscoveryManager] Topology snapshot [ver=36, servers=7, clients=5, CPUs=128, heap=160.0GB] > [15:10:05,702][INFO][exchange-worker-#62%statement_grid%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=36, minorTopVer=0], crd=false, evt=NODE_JOINED, evtNode=979cf868-1c37-424a-9ad1-12db501f32ef, customEvt=null, allowMerge=true] > [15:10:05,702][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=36, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=36, minorTopVer=0], err=null] > [15:10:05,702][INFO][exchange-worker-#62%statement_grid%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=36, minorTopVer=0], crd=false] > [15:10:05,703][INFO][exchange-worker-#62%statement_grid%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=36, minorTopVer=0], evt=NODE_JOINED, node=979cf868-1c37-424a-9ad1-12db501f32ef] > [15:10:08,706][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Cancelled rebalancing from all nodes [topology=AffinityTopologyVersion [topVer=35, minorTopVer=0]] > [15:10:08,707][INFO][exchange-worker-#62%statement_grid%][GridCachePartitionExchangeManager] Rebalancing scheduled [order=[statementp]] > [15:10:08,707][INFO][exchange-worker-#62%statement_grid%][GridCachePartitionExchangeManager] Rebalancing started [top=null, evt=NODE_JOINED, node=a8be3c14-9add-48c3-b099-3fd304cfdbf4] > [15:10:08,707][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=2f6bde48-ffb5-4815-bd32-df4e57dc13e0, partitionsCount=18, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > [15:10:08,707][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=35d01141-4dce-47dd-adf6-a4f3b2bb9da9, partitionsCount=15, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > [15:10:08,708][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=b3a8be53-e61f-4023-a906-a265923837ba, partitionsCount=15, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > [15:10:08,708][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=f825cb4e-7dcc-405f-a40d-c1dc1a3ade5a, partitionsCount=12, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > [15:10:08,708][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=4ae1db91-8b88-4180-a84b-127a303959e9, partitionsCount=11, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > [15:10:08,708][INFO][exchange-worker-#62%statement_grid%][GridDhtPartitionDemander] Starting rebalancing [mode=ASYNC, fromNode=7c286481-7638-49e4-8c68-fa6aa65d8b76, partitionsCount=18, topology=AffinityTopologyVersion [topVer=36, minorTopVer=0], updateSeq=-1754630006] > so in clusters with a big amount of data and the frequent client left/join events this means that a new server will never receive its partitions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)