Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 65BA8200CF7 for ; Tue, 5 Sep 2017 08:12:31 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6417516553D; Tue, 5 Sep 2017 06:12:31 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1016116553B for ; Tue, 5 Sep 2017 08:12:29 +0200 (CEST) Received: (qmail 3772 invoked by uid 500); 5 Sep 2017 06:12:28 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 3763 invoked by uid 99); 5 Sep 2017 06:12:28 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Sep 2017 06:12:28 +0000 Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id E91121A00DF for ; Tue, 5 Sep 2017 06:12:26 +0000 (UTC) Received: by mail-qk0-f177.google.com with SMTP id z143so8476929qkb.3 for ; Mon, 04 Sep 2017 23:12:25 -0700 (PDT) X-Gm-Message-State: AHPjjUjlFfE/8dLf56KjA10b/3F2a5ZX23aSPvoTbNZxWQDvvijpfADk JlFAcuiTUHRjpHcHSiWhU9Y+we13Vy1a X-Google-Smtp-Source: ADKCNb4e3g3H6sqJcMeJ/9OKxm9WG750EGsFExuWEAX/gZM0Dk2C2Ola1VHdOSVRbmcGtgWT5u8cHoOohjt/vnr0HjQ= X-Received: by 10.55.177.195 with SMTP id a186mr3666367qkf.78.1504591944105; Mon, 04 Sep 2017 23:12:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.34.6 with HTTP; Mon, 4 Sep 2017 23:11:42 -0700 (PDT) In-Reply-To: <8d16174bb59ca141f0b5abd9b5e91693@mail.gmail.com> References: <48eb5f43240b9a2dc7f4c10999362c9d@mail.gmail.com> <650b1bd91c7d0ed4d3d516bbbff07933@mail.gmail.com> <98f4b12ef9f3165cef8ac5ce269a4134@mail.gmail.com> <26ffcf7527aa580081464298419a40d8@mail.gmail.com> <8d16174bb59ca141f0b5abd9b5e91693@mail.gmail.com> From: Dmitriy Setrakyan Date: Mon, 4 Sep 2017 23:11:42 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Specifying location of persistent storage location To: user Content-Type: multipart/alternative; boundary="94eb2c06137a5c84e705586b1c99" archived-at: Tue, 05 Sep 2017 06:12:31 -0000 --94eb2c06137a5c84e705586b1c99 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Sep 4, 2017 at 8:40 PM, Raymond Wilson wrote: > Thanks. > > > > I get the utility of specifying the network address to bind to; I=E2=80= =99m not > convinced using that to derive the name of the internal data store is a > good idea! J > For instance, what if you have to move a persistent data store to a > different server? Or are you saying everybody sets LocalHost or 120.0.0.1 > to ensure the folder name is always essentially local host? > I think what you are asking about is a database backup or a snapshot. Ignite does not support it out of the box, but you may wish to look at the 3rd party solutions, e.g. the one provided by GridGain - https://docs.gridgain.com/docs/data-snapshots > > > *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] > *Sent:* Tuesday, September 5, 2017 3:09 PM > *To:* user > > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > On Mon, Sep 4, 2017 at 6:07 PM, Raymond Wilson > wrote: > > Dmitriy, > > > > I set up an XML file based on the default one and added the two elements > you noted. > > > > However, this has brought up an issue in that the XML file and an > IgniteConfiguration instance can=E2=80=99t both be provided to the Igniti= on.Start() > call. So I changed it to use the DiscoverSPI aspect of IgniteConfiguratio= n > and set LocalAddress to =E2=80=9C127.0.0.1=E2=80=9D and LocalPort to 4750= 0. > > > > This did change the name of the persistence folder to be =E2=80=9C127_0_0= _1_47500=E2=80=9D > as you suggested. > > > > While this resolves my current issue with the folder name changing, it > still seems fragile as network configuration aspects of the server Ignite > is running on have a direct impact on an internal aspect of its > configuration (ie: the location where to store the persisted data). A DHC= P > IP lease renewal or an internal DNS domain change or an internal IT > department change to using IPv6 addressing (among other things) could cau= se > problems when a node restarts and decides the location of its data is > different. > > > > Do you know how GridGain manage this in their enterprise deployments usin= g > persistence? > > > > I am glad the issue is resolved. By default, Ignite will bind to all the > local network interfaces, and if they are provided in different order, it > may create the situation you witnessed. > > > > All enterprise users explicitly specify which network address to bind to, > just like you did. This helps avoid any kind of magic in production. > > > > > > > > > > Thanks, > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] > *Sent:* Tuesday, September 5, 2017 11:41 AM > > > *To:* user > *Cc:* Raymond Wilson > *Subject:* Re: Specifying location of persistent storage location > > > > > > On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson > wrote: > > Hi, > > > > It=E2=80=99s possible this could cause change in the folder name, though = I do not > think this is an issue in my case. Below are three different folder names= I > have seen. All use the same port number, but differ in terms of the IPV6 > address (I have also seen variations where the IPv6 address is absent in > the folder name). > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_ > 50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > > > , > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_ > 8005_b03d_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_38b4_1_858c_ > f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_47500 > > > > I start the nodes in my local setup in a well defined order so I would > expect the port to be the same. I did once start a second instance by > mistake and did see the port number incremented in the folder name. > > > > Are you suggesting the two changes you note below will result in the same > folder name being chosen every time, unlike above? > > > > > > Yes, exactly. My suggestions will ensure that you explicitly bind to the > same address every time. > > > > > > > > > > > > Thanks, > > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] > *Sent:* Tuesday, September 5, 2017 11:17 AM > *To:* user > *Cc:* Raymond Wilson > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson > wrote: > > Hi, > > > > I definitely have not had more than one server node running at the same > time (though there have been more than one client node running on the sam= e > machine). > > > > I suspect what is happening is that one or more of the network interfaces > on the machine can have their address change dynamically. What I thought = of > as a GUID is actually (I think) an IPv6 address attached to one of the > interfaces. This aspect of the folder name tends to come and go. > > > > You can see from the folder names below that there are quite a number of > addresses involved. This seems to be fragile (and I certainly see the nam= e > of this folder changing frequently), so I think being able to set it to > something concrete would be a good idea. > > > > > > I think I understand what is happening. Ignite starts off with a default > port, and then starts incrementing it with every new node started on the > same host. Perhaps you start server and client nodes in different order > sometimes which causes server to bind to a different port. > > > > To make sure that your server node binds to the same port all the time, > you should try specifying it explicitly in the server node configuration, > like so (forgive me if this snippet does not compile): > > > > > > > > * class=3D"org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > * > > > > Please make sure that the client nodes either don't have any port > configured, or have a different port configured. > > > > You should also make sure that Ignite always binds to the desired local > interface on client and server nodes, by specifying IgniteConfiguration.s= etLocalHost(...) > property, or like so in XML: > > > > ** > > > > If my theory is correct, Ignite should make sure that the clients and > servers cannot theoretically bind to the same port. I will double check i= t > with the community and file a ticket if needed. > > > > > > > --94eb2c06137a5c84e705586b1c99 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Mon, Sep 4, 2017 at 8:40 PM, Raymond Wilson <raymond_wilso= n@trimble.com> wrote:

Thanks.

=C2=A0

I get the utility of specifying the network address to bind to; I=E2= =80=99m not convinced using that to derive the name of the internal data st= ore is a good idea! J

=C2=A0For instance, what if you have to move a persis= tent data store to a different server? Or are you saying everybody sets Loc= alHost or 120.0.0.1 to ensure the folder name is always essentially local h= ost?


I think what yo= u are asking about is a database backup or a snapshot. Ignite does not supp= ort it out of the box, but you may wish to look at the 3rd party solutions,= e.g. the one provided by GridGain -=C2=A0https://docs.gridgain.com/docs/data-snapshots<= /div>

=C2=A0

=C2=A0

From: Dmitriy Setrakyan = [mailto:dsetraky= an@apache.org]
Sent: Tuesday, September 5, 2017 3:09 PM
<= b>To: user <user@ignite.apache.org>

<= br>Subject: Re: Specifying location of persistent storage location

=C2=A0=

=C2=A0

=C2= =A0

On Mon, Sep 4, 2017 at 6:07 PM, Raymond = Wilson <= raymond_wilson@trimble.com> wrote:

Dmitriy,

=C2=A0

I set = up an XML file based on the default one and added the two elements you note= d.

=C2=A0

However, this has brough= t up an issue in that the XML file and an IgniteConfiguration instance can= =E2=80=99t both be provided to the Ignition.Start() call. So I changed it t= o use the DiscoverSPI aspect of IgniteConfiguration and set LocalAddress to= =E2=80=9C127.0.0.1=E2=80=9D and LocalPort to 47500.

= =C2=A0

This did change the name of the persistence fold= er to be =E2=80=9C127_0_0_1_47500=E2=80=9D as you suggested.

=C2=A0

While this resolves my current issue with = the folder name changing, it still seems fragile as network configuration a= spects of the server Ignite is running on have a direct impact on an intern= al aspect of its configuration (ie: the location where to store the persist= ed data). A DHCP IP lease renewal or an internal DNS domain change or an in= ternal IT department change to using IPv6 addressing (among other things) c= ould cause problems when a node restarts and decides the location of its da= ta is different.

=C2=A0

Do you know= how GridGain manage this in their enterprise deployments using persistence= ?

=C2=A0

=

I am glad the issue is resolved. By defau= lt, Ignite will bind to all the local network interfaces, and if they are p= rovided in different order, it may create the situation you witnessed.

<= /div>

=C2=A0

All enterprise users explicitly specify which network address to bind to,= just like you did. This helps avoid any kind of magic in production.

=C2=A0

=C2=A0

=C2=A0

=C2=A0

Thanks,
Raymond.=

=C2=A0

From: Dmitriy Setrakyan [m= ailto:dsetrakyan= @apache.org]
Sent: Tuesday, September 5, 2017 11:41 AM


To: user <user@ignite.apache.org&g= t;
Cc: Raymond Wilson <raymond_wilson@trimble.com>
Subject:<= /b> Re: Specifying location of persistent storage location

<= div>

=C2=A0

= =C2=A0

On Mon, Sep 4, 2017 at 4:28 PM, Raymo= nd Wilson <raymond_wilson@trimble.com> wrote:

Hi,

=C2=A0

It=E2=80=99s possi= ble this could cause change in the folder name, though I do not think this = is an issue in my case. Below are three different folder names I have seen.= All use the same port number, but differ in terms of the IPV6 address (I h= ave also seen variations where the IPv6 address is absent in the folder nam= e).

0_0_0_0_0_0_0_1_1= 0_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_1= 68_3_1_2406_e007_9e5_1_9cc8_92bc_50c9_6794_2406_e007_9e5_1_c= 5d8_af4b_55b2_582a_47500=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 , =

0_0_0_0_0_0_0_1_10_0_75_1_1= 0_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_= 1_2406_e007_9e5_1_a58c_2f32_8005_b03d_2406_e007_9e5_1_c5d8_af4b_5= 5b2_582a_47500

0_0_0_0_0_0_0_1_10_0_75_1_1= 0_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_= 1_2406_e007_38b4_1_858c_f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b= _55b2_582a_47500

=C2=A0

I start the nodes = in my local setup in a well defined order so I would expect the port to be = the same. I did once start a second instance by mistake and did see the por= t number incremented in the folder name.

<= span style=3D"font-size:11pt;font-family:Calibri,sans-serif">=C2=A0<= /p>

Are you suggesting the two changes you note below will result = in the same folder name being chosen every time, unlike above?

=C2=A0

=C2=A0

Yes, exactl= y. My suggestions will ensure that you explicitly bind to the same address = every time.

=C2=A0

=C2=A0

=C2=A0

=C2=A0

=C2=A0

Thanks,

Raymond.=

=C2=A0

From: Dmitriy Setrakyan [mailto:<= /span>dsetrakyan@apache.org] Sent: Tuesday, September 5, 2017 11:17 AM
To: user <<= /span>user@ignite.apache.org= &g= t;
Cc: Raymond Wilson <raymond_wilson@trimble.com>
Subject: Re: Spec= ifying location of persistent storage location

=C2=A0

=C2=A0

=C2=A0

On Mon, Sep 4, 2017 at 3:37 PM,= Raymond Wilson <raymond_wilson@trimble.com> wrote:

Hi,

=C2=A0

I = definitely have not had more than one server node running at the same time = (though there have been more than one client node running on the same machi= ne).

=C2=A0

I suspect what is happe= ning is that one or more of the network interfaces on the machine can have = their address change dynamically. What I thought of as a GUID is actually (= I think) an IPv6 address attached to one of the interfaces. This aspect of = the folder name tends to come and go.

=C2=A0

=

You can see from the folder names below that there are quite a nu= mber of addresses involved. This seems to be fragile (and I certainly see t= he name of this folder changing frequently), so I think being able to set i= t to something concrete would be a good idea.

=C2=A0

=C2=A0<= /p>

I think I understand what is happening= . Ignite starts off with a default port, and then starts incrementing it wi= th every new node started on the same host. Perhaps you start server and cl= ient nodes in different order sometimes which causes server to bind to a di= fferent port.=C2=A0

=C2=A0

To make sure that your server node binds to the s= ame port all the time, you should try specifying it explicitly in the serve= r node configuration, like so (forgive me if this snippet does not compile)= :

=C2=A0

<property name=3D"discoverySpi">=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<bean class=3D"org.apache.ignite= .spi.discovery.tcp.TcpDiscoverySpi">
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<property name=3D"localPort" va= lue=3D"40123"/>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0</bean= >
</property>

=C2=A0

Please make sure that the = client nodes either don't have any port configured, or have a different= port configured.

=C2=A0

You should also make sure that Ignite always b= inds to the desired local interface on client and server nodes, by specifyi= ng IgniteConfiguration.setLocalHost(...) property, or like so in XML:<= /p>

=C2=A0

<property name=3D"localHost" value=3D"m= y.local.ip.address"/>

=C2=A0

If my theory is correc= t, Ignite should make sure that the clients and servers cannot theoreticall= y bind to the same port. I will double check it with the community and file= a ticket if needed.

=C2=A0

<= /div>

=C2=A0

=C2=A0


--94eb2c06137a5c84e705586b1c99--