From dev-return-30513-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Tue Jan 30 22:55:26 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 4205018061A for ; Tue, 30 Jan 2018 22:55:26 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 31965160C53; Tue, 30 Jan 2018 21:55:26 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 53C53160C2A for ; Tue, 30 Jan 2018 22:55:25 +0100 (CET) Received: (qmail 27571 invoked by uid 500); 30 Jan 2018 21:55:24 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 27559 invoked by uid 99); 30 Jan 2018 21:55:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jan 2018 21:55:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 364AFC0452 for ; Tue, 30 Jan 2018 21:55:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id q3211Z_gi3vA for ; Tue, 30 Jan 2018 21:55:21 +0000 (UTC) Received: from mail-it0-f54.google.com (mail-it0-f54.google.com [209.85.214.54]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C10105F6D2 for ; Tue, 30 Jan 2018 21:55:20 +0000 (UTC) Received: by mail-it0-f54.google.com with SMTP id u12so3916628ite.0 for ; Tue, 30 Jan 2018 13:55:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=Kcps0ZX11y9hImn3UKdhuaMOj4SduwscmKGpsC9vm4U=; b=jQZvgDWWSz5jf9c1SjJrRphQQWPbZ/3YfNdjKAZZDjbtIF+eacMwArHP8BKygNJLS+ NeugTBzyqz9bambBEVI/gzZpr6vfDmD5v177n9lLdivJUK7+sU5DXqxOMOPlmPiEhmm3 iT0kemYAwf3MklXVt6NxnRsqFpLP0ZWfYWIZ8QnVGBWDqd+1B/641NiK0+WXX6lbzNDw ONA90gfHV6MCBbGtbGseH5bPqVCsLxheaW8kruT3F2uZzh4yv0x+u2UhyfvlPx/KlChf uW1RIVFzH++5TuPdyVO1EKckYWgxsvaKznpKTNItUXIQWX+IJuBMZlKzmUyxIejOnr92 KZ+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Kcps0ZX11y9hImn3UKdhuaMOj4SduwscmKGpsC9vm4U=; b=O8abOXT/TA0p14Yq610A24j345OylaMRkB96tYpE4mbHcHdCBomVZpV06dE4h5xXkf +EO6gTB9h7m4ycmk7O7uzXHTRigaW4iRPGXVSU18JO1xG+kvp7ktTnwT+HWRywvH6uK2 jaKuUupzcmq5lsKmRIl0Fx0I8V5ZlUbptpJlF9T0UtH2iJRJgYBM5+sNru0oMsYsvOuP H4fwbpwePfHvZTFfX6ImS0td/0bLArLmJzUhi8vTEAnHRepDu+KfmG4ukspJEf4slrTG 64P3p2tjHfZp+mJtt7BC/n6TYv429sQwCoqe296e2E3lPAh91fvZkw/MD7xW0C3LwBsV O0Kg== X-Gm-Message-State: AKwxytcA7JgEjN8PWzC5oWOFsx9uawOQZU7nUMlwiVY9eywXPHr92km2 j9cQQPhAaqNvKLX/q4nJJACd7h8NUifK/kI6/FY= X-Google-Smtp-Source: AH8x224swl4GiqbsyFw9ssgAI+FzxANq80HlbZstASbSpghPwBgYV7D/d8AABArC9GrCaHocyG4zA85/lQqsyFSdw4Y= X-Received: by 10.36.37.2 with SMTP id g2mr31834440itg.129.1517349319872; Tue, 30 Jan 2018 13:55:19 -0800 (PST) MIME-Version: 1.0 References: <5CB36582-B159-46C3-9D15-D5A05261F806@apache.org> <5a704e78.d5452e0a.b3fb5.9392@mx.google.com> <02592541-D03C-4394-95D1-8ED021E36EFE@apache.org> In-Reply-To: From: Dmitry Pavlov Date: Tue, 30 Jan 2018 21:55:09 +0000 Message-ID: Subject: Re: IGNITE_HOME for persistence To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary="001a1143d1fc36193c0564056b3b" --001a1143d1fc36193c0564056b3b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Igniters, Why it can be serious issue for end users? As far as I know IGNITE_HOME is always set using start scripts, e.g. in ignite.sh. If Ignite is started using java code from IDE, it may be required to set storage directory or work or home directory. Until that, TEMP is convinient for development. IMO it is quite obvious that persistent store configuration has 2 methods setPersistentEnabled and setStorageDirectory, isn't it? Moreover all paths are logged at startup. +1 for warn user if PDS-enabled ignite instance is started without any explicit folder specification. But I can't find a reason to consider it blocker for 2.4. Sincerely, Dmitriy Pavlov =D0=B2=D1=82, 30 =D1=8F=D0=BD=D0=B2. 2018 =D0=B3. =D0=B2 23:55, Alexey Gonc= haruk : > Denis, Dmitriy, > > I do not think the behavior should be in any way specific to the case whe= n > persistence is enabled. We have quite a bit of other information (such as > binary marshaller metadata and marshaller class mappings) that are also > stored in the work folder and may be required after the cluster restart, > moreover, this has been so for years since ignite 1.0. > > If you think this is a blocker, please suggest what behavior we want to s= ee > in ignite 2.4. We cannot calculate IGNITE_HOME when Ignite is run in an > embedded mode (in other words, the Ignite distribution package structure = is > not preserved). Printing a warning in this case looks good enough for me. > > 2018-01-30 23:35 GMT+03:00 Denis Magda : > > > The temp dir is no way to go for the persistence. > > > > I would enforce setting paths for the storage and WAL files if > IGNITE_HOME > > can't be calculated and we end up placing the storage files under the > > temp.dir. > > > > Alex G., can you join this thread? Seems that there is an oversight on > our > > side that needs to be fixed in 2.4. > > > > =E2=80=94 > > Denis > > > > > On Jan 30, 2018, at 2:52 AM, Stanislav Lukyanov < > stanlukyanov@gmail.com> > > wrote: > > > > > > I checked the code handling the IGNITE_HOME and persistent storage > paths, > > > and here is what the algorithm looks like. > > > > > > For IGNITE_HOME the following is checked in order; if on any step a > > value is found then we use it. > > > - IgniteHome in IgniteConfiguration > > > - IGNITE_HOME system property > > > - IGNITE_HOME environment variable > > > - Current working directory (user.dir) and all its ancestors (all > > directories are checked to have =E2=80=9Cbin/=E2=80=9D and =E2=80=9Ccon= fig/=E2=80=9D) > > > - Class path entry containing ignite-core classes and all its ancesto= rs > > > > > > After that, the working directory will be created at one of the > > following paths > > > - WorkingDirectory in IgniteConfiguration, if set > > > - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously > > > - ${java.io.tmpdir}/work > > > > > > Persistent storage will be stored in the working directory, unless > > StoragePath are specified in the config > > > (same for WAL and WalPath). > > > > > > The issue here is that if we=E2=80=99ve ended up having persistent DB= in the > > working directory in the /tmp, > > > then persistence files will be cleared upon restart. > > > Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if > > IGNITE_HOME is not set, but that=E2=80=99s not the case. > > > > > > So, how about actually disallowing to run Ignite when IGNITE_HOME can= =E2=80=99t > > be calculated? Using /tmp for working > > > directory seems to be an obscure and potentially harmful scenario. > > > IgniteConfiguration=E2=80=99s documentation can also be adjusted to s= pecify > > actual steps used to find IgniteHome and WorkingDirectory > > > if they aren=E2=80=99t set explicitly. > > > Additionally, I=E2=80=99d suggest not to promote setting IGNITE_HOME = system > > property and environment variable > > > (e.g. let=E2=80=99s remove it from readmeio). IgniteConfiguration see= ms to be > > the most straightforward way to configure Ignite, > > > and system properties should be used as a backup plan when convenient= . > > > > > > WDYT? > > > > > > Thanks, > > > Stan > > > > > > From: Denis Magda > > > Sent: 30 =D1=8F=D0=BD=D0=B2=D0=B0=D1=80=D1=8F 2018 =D0=B3. 3:38 > > > To: dev@ignite.apache.org > > > Subject: Re: IGNITE_HOME for persistence > > > > > > No we don=E2=80=99t. I=E2=80=99ve never touched IGNITE_HOME variable = for any other > > purpose. > > > > > > As it was suggested, the reported should share the project to reprodu= ce > > his scenario. > > > > > > =E2=80=94 > > > Denis > > > > > >> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan > wrote: > > >> > > >> Igniters, > > >> > > >> I have just stumbled upon this post on SO: > > >> https://stackoverflow.com/questions/48434929/apache- > > ignite-persistent-storage > > >> > > >> Do we require IGNITE_HOME to be set if the persistence is enabled? I= f > > yes, > > >> do we check for it on startup? > > >> > > >> D. > > > > > > > > > > > --001a1143d1fc36193c0564056b3b--