Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 061F6200CF7 for ; Tue, 5 Sep 2017 05:41:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 048E8164CA4; Tue, 5 Sep 2017 03:41:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7E893164C9F for ; Tue, 5 Sep 2017 05:41:04 +0200 (CEST) Received: (qmail 69860 invoked by uid 500); 5 Sep 2017 03:40:58 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 69850 invoked by uid 99); 5 Sep 2017 03:40:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Sep 2017 03:40:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 101CED4347 for ; Tue, 5 Sep 2017 03:40:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.481 X-Spam-Level: ** X-Spam-Status: No, score=2.481 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=trimble-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id uVomrQO8R0Ey for ; Tue, 5 Sep 2017 03:40:51 +0000 (UTC) Received: from mail-qt0-f174.google.com (mail-qt0-f174.google.com [209.85.216.174]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id D0BE85FACE for ; Tue, 5 Sep 2017 03:40:50 +0000 (UTC) Received: by mail-qt0-f174.google.com with SMTP id i50so8055913qtf.0 for ; Mon, 04 Sep 2017 20:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=trimble-com.20150623.gappssmtp.com; s=20150623; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to; bh=XU5RqpzIHxNehCl07bzjSe33WNk+o2RzFgRx4ft2F2w=; b=KMyoJe/EUGB2b/ByClu/i7v/mAzfOmnJnkH8NatiiFS3+PGDTGuB0j35OZcrSmraR7 l6ZEAsN1WlVVlUELAJixCyTYosZSMlJ3rcjq28BgyAvEQG8ylyh1DKn2/mjWuvBBZbrS S1i6bWg4uduu+ZKUNpjZDtio/0osSkLRFbYa8INd7mvoRHB0+oY8DB6k3uBdTwpHAhBO 2bCg3s4m8UDHE9Rt2j+juTz/1x5bM/cPvTpXWZZlvxmbBserxVw8Go/U6xlDWBnjPGsc hupQg9y4JYnDzQFTb8BShAqI6YwwTZBuxadLD+7Kn9pT4Uk/elJvhDCbjyz7YVYshO/v TAjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to; bh=XU5RqpzIHxNehCl07bzjSe33WNk+o2RzFgRx4ft2F2w=; b=hQ3cycHbdu2oVoqbB8xfOpwIjYKn8AXYGnTyUPxzjh83qNnQx9zdgMA44x32XF1g64 wRXUPSE25lyHRndTq6STY5ByWQpX8dDWYv/BwQdYVNkpVpPdTtKBaPQTkKFSTu7YubUL n03dYtBSOKRGb8sSDnKfDJfMaYrFjtMKK/MEfqlNiCeGQ1uB8wysgchjs/obMtucVbi9 2mR0Tif8csnSLmcBA9bvb9niXNaY77D+9GvAi0eL442bd25VgLIyve5oo3XJ+JdNXPuy h9dIgz5ORPO/PNPEoNuVWcc8R2ilq3HAkLuqD3p2BV5i08JYNAKL1j9Wmgntyh34kXx4 0cuA== X-Gm-Message-State: AHPjjUiJ2QF00l/1Ty3lyU/Xuh8dS0IVRjtueqHFJWjUDoAO+bbTflQD po+czZwELRi3jOIOdxRU6Lc2lsgHKxu5c0I= X-Google-Smtp-Source: ADKCNb458QFpaIwRnd4Lavm6IUw8lXY5SA//sNLn9fiBzVfUxBCLu+TPBkrJxT1RlQwXQru/rub7E/iWqwsbhdIAvJQ= X-Received: by 10.200.14.4 with SMTP id a4mr3361986qti.1.1504582843550; Mon, 04 Sep 2017 20:40:43 -0700 (PDT) From: Raymond Wilson References: <48eb5f43240b9a2dc7f4c10999362c9d@mail.gmail.com> <650b1bd91c7d0ed4d3d516bbbff07933@mail.gmail.com> <98f4b12ef9f3165cef8ac5ce269a4134@mail.gmail.com> <26ffcf7527aa580081464298419a40d8@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQF00FljkSSCE8lpFoGQipuBV4OOSwHkupPZAg9EcpEA0aoPLAGXXhl4Ab1bmaUBpfhG5gIYzA0fApdQWlQCvDpWjAGBMADWAn1Fk9kCik+jmqKkpFJg Date: Tue, 5 Sep 2017 15:40:37 +1200 Message-ID: <8d16174bb59ca141f0b5abd9b5e91693@mail.gmail.com> Subject: RE: Specifying location of persistent storage location To: user@ignite.apache.org Content-Type: multipart/alternative; boundary="089e0822a2b8ed9bb9055868fdf1" archived-at: Tue, 05 Sep 2017 03:41:06 -0000 --089e0822a2b8ed9bb9055868fdf1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks. I get the utility of specifying the network address to bind to; I=E2=80=99m= not convinced using that to derive the name of the internal data store is a good idea! J For instance, what if you have to move a persistent data store to a different server? Or are you saying everybody sets LocalHost or 120.0.0.1 to ensure the folder name is always essentially local host? *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] *Sent:* Tuesday, September 5, 2017 3:09 PM *To:* user *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 6:07 PM, Raymond Wilson wrote: Dmitriy, I set up an XML file based on the default one and added the two elements you noted. However, this has brought up an issue in that the XML file and an IgniteConfiguration instance can=E2=80=99t both be provided to the Ignition= .Start() call. So I changed it to use the DiscoverSPI aspect of IgniteConfiguration and set LocalAddress to =E2=80=9C127.0.0.1=E2=80=9D and LocalPort to 47500. This did change the name of the persistence folder to be =E2=80=9C127_0_0_1= _47500=E2=80=9D as you suggested. While this resolves my current issue with the folder name changing, it still seems fragile as network configuration aspects of the server Ignite is running on have a direct impact on an internal aspect of its configuration (ie: the location where to store the persisted data). A DHCP IP lease renewal or an internal DNS domain change or an internal IT department change to using IPv6 addressing (among other things) could cause problems when a node restarts and decides the location of its data is different. Do you know how GridGain manage this in their enterprise deployments using persistence? I am glad the issue is resolved. By default, Ignite will bind to all the local network interfaces, and if they are provided in different order, it may create the situation you witnessed. All enterprise users explicitly specify which network address to bind to, just like you did. This helps avoid any kind of magic in production. Thanks, Raymond. *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] *Sent:* Tuesday, September 5, 2017 11:41 AM *To:* user *Cc:* Raymond Wilson *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson wrote: Hi, It=E2=80=99s possible this could cause change in the folder name, though I = do not think this is an issue in my case. Below are three different folder names I have seen. All use the same port number, but differ in terms of the IPV6 address (I have also seen variations where the IPv6 address is absent in the folder name). 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_2= 7_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_50c9_6794_2406_e007_9e5_1_c5d8_af4b= _55b2_582a_47500 , 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_2= 7_192_168_3_1_2406_e007_9e5_1_a58c_2f32_8005_b03d_2406_e007_9e5_1_c5d8_af4b= _55b2_582a_47500 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_2= 7_192_168_3_1_2406_e007_38b4_1_858c_f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af= 4b_55b2_582a_47500 I start the nodes in my local setup in a well defined order so I would expect the port to be the same. I did once start a second instance by mistake and did see the port number incremented in the folder name. Are you suggesting the two changes you note below will result in the same folder name being chosen every time, unlike above? Yes, exactly. My suggestions will ensure that you explicitly bind to the same address every time. Thanks, Raymond. *From:* Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] *Sent:* Tuesday, September 5, 2017 11:17 AM *To:* user *Cc:* Raymond Wilson *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson wrote: Hi, I definitely have not had more than one server node running at the same time (though there have been more than one client node running on the same machine). I suspect what is happening is that one or more of the network interfaces on the machine can have their address change dynamically. What I thought of as a GUID is actually (I think) an IPv6 address attached to one of the interfaces. This aspect of the folder name tends to come and go. You can see from the folder names below that there are quite a number of addresses involved. This seems to be fragile (and I certainly see the name of this folder changing frequently), so I think being able to set it to something concrete would be a good idea. I think I understand what is happening. Ignite starts off with a default port, and then starts incrementing it with every new node started on the same host. Perhaps you start server and client nodes in different order sometimes which causes server to bind to a different port. To make sure that your server node binds to the same port all the time, you should try specifying it explicitly in the server node configuration, like so (forgive me if this snippet does not compile): * * Please make sure that the client nodes either don't have any port configured, or have a different port configured. You should also make sure that Ignite always binds to the desired local interface on client and server nodes, by specifying IgniteConfiguration.setLocalHost(...) property, or like so in XML: ** If my theory is correct, Ignite should make sure that the clients and servers cannot theoretically bind to the same port. I will double check it with the community and file a ticket if needed. --089e0822a2b8ed9bb9055868fdf1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Thanks.

=C2=A0

I get the utility = of specifying the network address to bind to; I=E2=80=99m not convinced usi= ng that to derive the name of the internal data store is a good idea! J

=C2=A0

For inst= ance, what if you have to move a persistent data store to a different serve= r? Or are you saying everybody sets LocalHost or 120.0.0.1 to ensure the fo= lder name is always essentially local host?

=C2=A0

From: Dmitriy S= etrakyan [mailto:dsetrakyan@apache= .org]
Sent: Tuesday, September 5, 2017 3:09 PM
To:= user <user@ignite.apache.org<= /a>>
Subject: Re: Specifying location of persistent storage lo= cation

=C2=A0

=C2=A0

=C2=A0

On Mon, Sep 4, 2017 at 6:07 PM, Raymond Wilson <raymond_wilson@trimble.com&= gt; wrote:

Dmitriy,

=C2=A0

I set up= an XML file based on the default one and added the two elements you noted.=

=C2=A0

However, this has brought u= p an issue in that the XML file and an IgniteConfiguration instance can=E2= =80=99t both be provided to the Ignition.Start() call. So I changed it to u= se the DiscoverSPI aspect of IgniteConfiguration and set LocalAddress to = =E2=80=9C127.0.0.1=E2=80=9D and LocalPort to 47500.

=C2= =A0

This did change the name of the persistence folder = to be =E2=80=9C127_0_0_1_47500=E2=80=9D as you suggested.

=C2=A0

While this resolves my current issue with the= folder name changing, it still seems fragile as network configuration aspe= cts of the server Ignite is running on have a direct impact on an internal = aspect of its configuration (ie: the location where to store the persisted = data). A DHCP IP lease renewal or an internal DNS domain change or an inter= nal IT department change to using IPv6 addressing (among other things) coul= d cause problems when a node restarts and decides the location of its data = is different.

=C2=A0

Do you know ho= w GridGain manage this in their enterprise deployments using persistence?

=C2=A0

I am glad the issue is resolved. By default,= Ignite will bind to all the local network interfaces, and if they are prov= ided in different order, it may create the situation you witnessed.

=C2=A0

A= ll enterprise users explicitly specify which network address to bind to, ju= st like you did. This helps avoid any kind of magic in production.

=C2=A0

= =C2=A0

=C2=A0

=C2=A0

<= p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-al= t:auto">Thanks,
Raymond.

=C2=A0

From: Dmitriy Setrakyan [mailto:dsetrakyan@apache.org]
Sent: Tuesday, September 5, 2017 11:41 AM


To: user <user@ignite.apache.org>
Cc: Raymond Wilson = <raymond= _wilson@trimble.com>
Subject: Re: Specifying location of p= ersistent storage location

=C2=A0

=C2=A0

On Mon, Sep 4, 2017 at 4:28 PM, Raym= ond Wilson <raymond_wilson@trimble.com> wrote:

Hi,

=C2=A0

It=E2=80=99s pos= sible this could cause change in the folder name, though I do not think thi= s is an issue in my case. Below are three different folder names I have see= n. All use the same port number, but differ in terms of the IPV6 address (I= have also seen variations where the IPv6 address is absent in the folder n= ame).

0_0_0_0_0_0_0_1_10_0_75_1_10_= 3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1= _9cc8_92bc_50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ,

0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_1= 68_121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_8005_b03d_240= 6_e007_9e5_1_c5d8_af4b_55b2_582a_47500

0_0_0_0_0_0_0_1_1= 0_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_1_240= 6_e007_38b4_1_858c_f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_4750= 0

=C2=A0

I start the nodes in my local set= up in a well defined order so I would expect the port to be the same. I did= once start a second instance by mistake and did see the port number increm= ented in the folder name.

=C2=A0

Ar= e you suggesting the two changes you note below will result in the same fol= der name being chosen every time, unlike above?

=C2=A0

=C2=A0

Yes, exactly. My suggestions will ensure that you explicitly bind to= the same address every time.

=C2=A0

<= p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-al= t:auto">=C2=A0

=C2=A0

=C2=A0<= /p>

=C2=A0

Thank= s,

Raymond.

=C2=A0

From: Dmitriy Setrakyan [mailto:dsetrakyan@apache.org= ]
Sent: Tuesday, September 5, 2017 11:17 AM
To: user = <use= r@ignite.apache.org>
Cc: Raymond Wilson <raymond_w= ilson@trimble.com>
Subject: Re: Specifying locatio= n of persistent storage location

=C2=A0

=C2=A0

=C2=A0

On Mon, Sep 4, 2017 at 3:3= 7 PM, Raymond Wilson <raymond_wilson@trimble.com> wrote:

Hi,

=C2=A0

I definitely have not had more than one server node running at the same= time (though there have been more than one client node running on the same= machine).

=C2=A0

I suspect what is= happening is that one or more of the network interfaces on the machine can= have their address change dynamically. What I thought of as a GUID is actu= ally (I think) an IPv6 address attached to one of the interfaces. This aspe= ct of the folder name tends to come and go.

=C2=A0

You can see from the folder names below that there are qui= te a number of addresses involved. This seems to be fragile (and I certainl= y see the name of this folder changing frequently), so I think being able t= o set it to something concrete would be a good idea.

= =C2=A0

=C2=A0

I think I understand what is happening. Ignite starts off with a defaul= t port, and then starts incrementing it with every new node started on the = same host. Perhaps you start server and client nodes in different order som= etimes which causes server to bind to a different port.=C2=A0

=C2=A0

To make sure that your server node = binds to the same port all the time, you should try specifying it explicitl= y in the server node configuration, like so (forgive me if this snippet doe= s not compile):

=C2=A0

<= p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-al= t:auto"><property name=3D"discoverySpi">
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0<bean class=3D"org.apache.ignite.spi.discov= ery.tcp.TcpDiscoverySpi">
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0<property name=3D"localPort" value=3D"40123&quo= t;/>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0</bean>
</property= >

=C2=A0

Please make sure that the client nodes either don't have any port conf= igured, or have a different port configured.

=C2=A0<= /p>

You should also make sure that Ignite always bi= nds to the desired local interface on client and server nodes, by specifyin= g IgniteConfiguration.setLocalHost(...) property, or like so in XML:

=C2=A0

<property nam= e=3D"localHost" value=3D"my.local.ip.address"/><= /p>

=C2=A0

If my the= ory is correct, Ignite should make sure that the clients and servers cannot= theoretically bind to the same port. I will double check it with the commu= nity and file a ticket if needed.

=C2=A0

=C2=A0<= /p>

=C2=A0

--089e0822a2b8ed9bb9055868fdf1--