From dev-return-49651-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Thu Feb 6 13:11:02 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id C243818064E for ; Thu, 6 Feb 2020 14:11:01 +0100 (CET) Received: (qmail 69353 invoked by uid 500); 6 Feb 2020 13:11:01 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 69333 invoked by uid 99); 6 Feb 2020 13:11:00 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Feb 2020 13:11:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3ACCF180643 for ; Thu, 6 Feb 2020 13:11:00 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.249 X-Spam-Level: * X-Spam-Status: No, score=1.249 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_REPLY=1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id mCTHtJakOrd5 for ; Thu, 6 Feb 2020 13:10:57 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.217.41; helo=mail-vs1-f41.google.com; envelope-from=vladsz83@gmail.com; receiver= Received: from mail-vs1-f41.google.com (mail-vs1-f41.google.com [209.85.217.41]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 989E8BB806 for ; Thu, 6 Feb 2020 13:10:57 +0000 (UTC) Received: by mail-vs1-f41.google.com with SMTP id p14so3703335vsq.6 for ; Thu, 06 Feb 2020 05:10:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=/3DvsR5O3D31+37EdpTlYnWh93SneVkFYY2RTRJUQPo=; b=NqOISuVuyqRuKf5ky8z98p2VyJHaOul0Q3PXPrG2ojnwohrz2P+3iOyflZd6SCB9Q5 ghp8VwkHgzoQMSBASTZWnCaHlA1U7Umymz+av4GpqPipLSxsLbci9HCQEjdizLTSbJR/ 9ONnEGYAUfEA9Ukb+m2ikrr407+DFHgd9n6gq43YnNi8WQKLHTHgz2vSBYS2ujX86FYH 0eJ//LCmLK8XA+kJcI+UlrUyx/r0jK0mTEiBN/OH3Gq4zuaQhhu+ruKNle9v3F01ZGsM rnxd1JezDS6/2pAQ9fXlXf+JRNPbZEKrwbSnhGIi3jrYBFe5xyxQCDt5MeM9Z1BLaQce DSGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=/3DvsR5O3D31+37EdpTlYnWh93SneVkFYY2RTRJUQPo=; b=MsUAzYV8o2zkAjQmFEqZ405ENoaa0c3LM3CMXrIjCU/YUaS6G93n32G26Iw1f38n6s Kpju5DtHv/4z4eeH/WiKOAJADqY7Gh8xV8ShpJ3lKhAgIGlj3EnP7PDGOtXZ37j8ueZg EkCx0vyfkpwE/w13Urya/sfGyAjah/QXxX77CwO9NRWLJnr5nQj4+e+2mE2vCtmLa2KF G/yafF2QD/kQV8N+7fIuPUlJA53dDhK4l6w9kdS3fTRJRXav1Iw4jFgkwn+8qm4+L4T3 WZAedhiyengLQ6GkHZl9LeG3LskiZSX95/fW+xppn6vKJPQsFfHilB1musOla/VsXsaz cimA== X-Gm-Message-State: APjAAAUNpZ6SSNhPnVtfO9DfK93itboxZXx6YBk49XSMec5LbUNKsmo7 hneLJnbkaBlAEEXqJNe8hir+Qgk2WksLF+BEJZonVXoWPVI= X-Google-Smtp-Source: APXvYqyUfykZSFch+p5jN5JQyArTZr6qrbsulU1O5kIExbXypHKp3YvS6vukO4qKNCxObQQQnba3mRXKUrMmDd3RQjs= X-Received: by 2002:a05:6102:38b:: with SMTP id m11mr1674550vsq.187.1580994651052; Thu, 06 Feb 2020 05:10:51 -0800 (PST) MIME-Version: 1.0 References: <3E3CDD4B-D72A-4C5A-8134-0156D097032F@gmail.com> <19F0AF2A-DEDE-4F32-B1D2-BE305D15A1B4@gmail.com> In-Reply-To: From: Vladimir Steshin Date: Thu, 6 Feb 2020 16:10:40 +0300 Message-ID: Subject: Re: Data vanished from cluster after INACTIVE/ACTIVE switch To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary="000000000000915dd7059de800c7" --000000000000915dd7059de800c7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Denis, Alexei, Regarding usage of flag org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE [1] When enabled, I think the following test should work. But it fails. //---------------------------------------------------------------- @Test public void testDataPresent() throws Exception { System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true"); IgniteEx i =3D startGrid(0); assertFalse( i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfigu= ration() .isPersistenceEnabled() ); String name =3D "non-persistent-cache"; i.createCache(name).put(1L, 1L); assertEquals(1L, i.cache(name).get(1L)); i.cluster().state(ClusterState.INACTIVE); i.cluster().state(ClusterState.ACTIVE); assertEquals(1L, i.cache(name).get(1L)); //Assertion error here! } //---------------------------------------------------------------- Several notes: - IgniteCacheDatabaseSharedManager#reuseMemory is true - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown) is called with shutdown =3D=3D false - PageMemoryNoStoreImpl#stop(booleam deallocate) is called with deallocate =3D=3D false But the cache from the test still has zero size after reactivation. Is flag [1] disabled by default because it is not implemented / doesn't work? Do we need to skip it in current ticket and rise new one? =D1=81=D1=80, 5 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 21:05, Denis = Magda : > I believe there might be a consistency-related reason why this flag was > disabled by default for caches that store data in Ignite native > persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on > this. > > As for the memory-only caches or caches backed up by a CacheStore such as > an RDBMS, enabling of the flag should be harmless. Once we do this, we'll > eliminate the need to load the data back into the cluster which can be a > time-consuming operation depending on the data volume. > > > - > Denis > > > On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin > wrote: > > > Denis, but why reuse-data-on-deactivate was disabled by default? There > > should be a reason for that. Any data consistency issues when node gets > > activated anew? We may use both solutions because the flag can be > switched > > off again. > > > > =D1=81=D1=80, 5 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 20:47, De= nis Magda : > > > > > Hi Vladimir, > > > > > > Yes, I'm suggesting us to enable this flag by > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVAT= E > > > default instead of introducing --force flag and showing any warnings. > > > > > > - > > > Denis > > > > > > > > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin > > > wrote: > > > > > > > Hello all. > > > > > > > > Denis, which changes exactly? In current implementation of ticket [= 2] > > > flag > > > > [1] is checked before requiring --force flag and showing any > warnings. > > Do > > > > we need to set reuse-memory-on-deactivate to true by default? > > > > > > > > [1] > > > > > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVAT= E > > > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614 > > > > > > > > > > > > =D0=B2=D1=82, 4 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 22:45= , Denis Magda : > > > > > > > > > That's the best solution for this scenario. Should we readjust th= e > > > > already > > > > > created ticket [1] suggesting to implement the changes of Alex > > > Scherbakov > > > > > instead? > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614 > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov < > > > > > alexey.scherbakoff@gmail.com> wrote: > > > > > > > > > > > Folks, > > > > > > > > > > > > For a long time we have a flag [1] > > > > > > > > > > > > It does almost what we want here. > > > > > > > > > > > > I suggest to make this behavior default and rework it to keep > data > > in > > > > > > memory as well (we already have special "recovery" mode for > > caches). > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVAT= E > > > > > > > > > > > > > > > > > > > > > > > > =D0=BF=D0=BD, 3 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 1= 8:47, Alexey Goncharuk < > > > > > alexey.goncharuk@gmail.com > > > > > > >: > > > > > > > > > > > > > I do not mind making this change if we explicitly and clearly > > > define > > > > > what > > > > > > > 'new inactive state' means. What should happen if a partition > is > > > lost > > > > > in > > > > > > > inactive state? What if such node joins back the cluster afte= r? > > > Etc. > > > > > > > > > > > > > > =D0=BF=D1=82, 31 =D1=8F=D0=BD=D0=B2. 2020 =D0=B3. =D0=B2 20:5= 7, Denis Magda : > > > > > > > > > > > > > > > Back up Ivan's opinion here. Initially, the > > > activation/deactivation > > > > > was > > > > > > > > created for the baseline topology designed for cases with > > native > > > > > > > > persistence. My thinking was that the mechanism itended to > > > prevent > > > > > data > > > > > > > > inconsistencies while nodes with data on the disk will be i= n > > the > > > > > > process > > > > > > > of > > > > > > > > joining the cluster. > > > > > > > > > > > > > > > > Artem, could you please update the docs bringing this to th= e > > > > > attention > > > > > > of > > > > > > > > the user community? > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615 > > > > > > > > > > > > > > > > AG, what if we don't purge data from memory at least for th= e > > > caches > > > > > not > > > > > > > > backed by the native persistence? Is this a big deal? We ca= n > > > > > certainly > > > > > > > put > > > > > > > > this off by my guts feel we'll return to this question soon= er > > or > > > > > later. > > > > > > > > > > > > > > > > - > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin < > > > > vololo100@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > For me it looks like some coincidence effect. I understan= d > > that > > > > we > > > > > > get > > > > > > > > > such behavior because deactivation works the same way as > for > > > > > > > > > persistent caches. Was cluster activation/deactivation > > designed > > > > and > > > > > > > > > described for in-memory caches? Current behavior sounds f= or > > me > > > a > > > > as > > > > > > > > > big risk. I expect a lot of upset users unexpectedly purg= ed > > all > > > > > their > > > > > > > > > data. > > > > > > > > > > > > > > > > > > =D0=BF=D1=82, 31 =D1=8F=D0=BD=D0=B2. 2020 =D0=B3. =D0=B2 = 00:00, Alexey Goncharuk < > > > > > > > > alexey.goncharuk@gmail.com > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > Because originally the sole purpose of deactivation was > > > > resource > > > > > > > > > > deallocation. > > > > > > > > > > > > > > > > > > > > =D1=87=D1=82, 30 =D1=8F=D0=BD=D0=B2. 2020 =D0=B3. =D0= =B2 22:13, Denis Magda < > > dmagda@apache.org > > > >: > > > > > > > > > > > > > > > > > > > > > Such a revelation for me that data is purged from RAM > if > > > > > someone > > > > > > > > > > > deactivates the cluster. Alex, do you remember why we > > > decided > > > > > to > > > > > > > > > implement > > > > > > > > > > > it this way initially? > > > > > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk < > > > > > > > > > > > alexey.goncharuk@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > I agree on CLI and JMX because those interfaces can > be > > > used > > > > > by > > > > > > a > > > > > > > > > system > > > > > > > > > > > > administrator and can be invoked by mistake. > > > > > > > > > > > > > > > > > > > > > > > > As for the Java API, personally, I find it strange = to > > add > > > > > > 'force' > > > > > > > > or > > > > > > > > > > > > 'confirm' flags to it because it is very unlikely > that > > > such > > > > > an > > > > > > > > > invocation > > > > > > > > > > > > is done by mistake. Such mistakes are caught during > the > > > > > testing > > > > > > > > > phase and > > > > > > > > > > > > developers will end up hard-coding 'true' as a flag > > > > anyways. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Best regards, > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Best regards, > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > --000000000000915dd7059de800c7--