Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 2E2E6200B29 for ; Wed, 15 Jun 2016 17:59:39 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 2CF0E160A4D; Wed, 15 Jun 2016 15:59:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 25676160A5D for ; Wed, 15 Jun 2016 17:59:37 +0200 (CEST) Received: (qmail 32597 invoked by uid 500); 15 Jun 2016 15:54:28 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 29106 invoked by uid 99); 15 Jun 2016 15:48:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 15:48:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 9A98BC1D32 for ; Wed, 15 Jun 2016 15:33:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.18 X-Spam-Level: * X-Spam-Status: No, score=1.18 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id N2SDFSm3noWo for ; Wed, 15 Jun 2016 15:33:13 +0000 (UTC) Received: from mail-qk0-f173.google.com (mail-qk0-f173.google.com [209.85.220.173]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 6ED4A5F1F7 for ; Wed, 15 Jun 2016 15:33:12 +0000 (UTC) Received: by mail-qk0-f173.google.com with SMTP id a186so23076302qkf.0 for ; Wed, 15 Jun 2016 08:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=JgOE9WitxQoKzrmdAp00NkdOs7DLBM+QjeZzRXKYUls=; b=he90uuBoJJ+T1d0FX6/ajJTHS+ySnhYS3i8U4MR5y/0xVySCTis8xhhXZnTpCrRqEq LZaN1O+n6QQ0f5HPu3Zv6lGgff+9hKRhbdqkjMJAGUcpYiKLQG4w3qYASh/ylrT2pMD6 KxNcyEnk8bW8muH7IdyI2ZDE7upapFJ8yCDVmGNHMzJ0/WmbE8ijY6zwZfXSmWQFm2i7 1H5GCe9M1smgGXuh6vLEL8cM9VITlDRGwfcGl9Wq8FZ8udwt/t2Hbg6I/QLnP3qBSlgz P4Uql+WyL6Su5Y2vAu4WNZ5/DV3L+CThp9hya/pNl6KhkFt6DmpuZJyduz1UUBssAYl3 FtGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=JgOE9WitxQoKzrmdAp00NkdOs7DLBM+QjeZzRXKYUls=; b=HhL+/XY3lgervAj5WPmWFzc/pDvV7FS+0K0limm87f0URzyWElI08r9SvFDbA1Uw4j OV9JoTlL05OY4b+P0fQ4GYKovQIeOmnVJoB8d5X1XTKD1iN4k4/IPMJIvh6XMA1XdcsU FRccgBDcsh32YgioyR2bbKE2MfdwskEx9rLBGAU4XxCZNS7j2yn0c+1gk0kEZUQqLfxs klggxH5Nyf8lhyAM3By20p2u0QpN6orP+LXa13eBpWch4O3ILxxgy/eYLsKnTfXlmbJs hW7GpMkywBdPAqugZK8yZbYHGuYiNXGy68gQXNXatP6kAhwqJO1aMLDkqtb2dEPKgtxq gMlA== X-Gm-Message-State: ALyK8tLqofNak1zx7hmQ9s9KMAlZm6YFkZxfrTYRj3/+KsNUNCHFC5JmxoklhClDRZ3D5WrPZ1unqzlikF9c+g== X-Received: by 10.55.160.211 with SMTP id j202mr27221052qke.0.1466004791424; Wed, 15 Jun 2016 08:33:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.37.72 with HTTP; Wed, 15 Jun 2016 08:33:10 -0700 (PDT) In-Reply-To: References: From: Alexei Scherbakov Date: Wed, 15 Jun 2016 18:33:10 +0300 Message-ID: Subject: Re: How deploy Ignite workers in a Spark cluster To: user Content-Type: multipart/alternative; boundary=001a114fadecd4fae5053552d66d archived-at: Wed, 15 Jun 2016 15:59:39 -0000 --001a114fadecd4fae5053552d66d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This example (slightly modified) works fine for me. https://gist.github.com/ascherbakoff/7c19da8ba568076221598d9d97e96b77 In logs I see a problem with multicast IP discovery. Check if the mulitcast is enabled on your machine or better use static IP discovery[1] [1] https://apacheignite.readme.io/docs/cluster-config#static-ip-based-discover= y 2016-06-15 18:06 GMT+03:00 Paolo Di Tommaso : > The version is 1.6.0#20160518-sha1:0b22c45b and the following is the > script I'm using. > > > https://github.com/pditommaso/gspark/blob/master/src/main/groovy/org/apac= he/ignite/examples/JavaIgniteSimpleApp.java > > > > Cheers, p > > > On Wed, Jun 15, 2016 at 5:00 PM, Alexei Scherbakov < > alexey.scherbakoff@gmail.com> wrote: > >> I don't think it's OK. >> >> Which Ingite's version do you use? >> >> 2016-06-15 15:35 GMT+03:00 Paolo Di Tommaso : >> >>> Great, now it works! Thanks a lot. >>> >>> >>> I have only a NPE during the application shutdown (you can find the >>> stack trace at this link ). Is this >>> normal? and in any case is there a way to avoid it? >>> >>> >>> Cheers, >>> Paolo >>> >>> >>> >>> On Wed, Jun 15, 2016 at 1:25 PM, Alexei Scherbakov < >>> alexey.scherbakoff@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> To automatically start Ignite nodes you must pass false parameter to >>>> 3-d IgniteContext argument like: >>>> >>>> // java >>>> SparcContext sc =3D ... >>>> new JavaIgniteContext<>(sc, new IgniteConfigProvider(), false);; >>>> >>>> or >>>> >>>> // scala >>>> SparcContext sc =3D ... >>>> new IgniteContext[String, String](sc,() =E2=87=92 configurationClo(), = false) >>>> >>>> 2016-06-15 13:31 GMT+03:00 Paolo Di Tommaso >>>> : >>>> >>>>> Hi all, >>>>> >>>>> I'm struggling deploying an Ignite application in a Spark (local) >>>>> cluster using the Embedded deploying described at this link >>>>> . >>>>> >>>>> >>>>> The documentation seems suggesting that Ignite workers are >>>>> automatically instantiated at runtime when submitting the Ignite app. >>>>> >>>>> Could you please confirm that this is the expected behaviour? >>>>> >>>>> >>>>> In my tests the when the application starts it simply hangs, reportin= g >>>>> this warning message: >>>>> >>>>> WARN org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Failed t= o >>>>> connect to any address from IP finder (will retry to join topology ev= ery 2 >>>>> secs): [/192.168.1.36:47500, /192.168.99.1:47500] >>>>> >>>>> It looks like there are not ignite daemons to which connect to. Also >>>>> inspecting the Spark worker log I'm unable to find any message produc= ed by >>>>> Ignite. I'm expecting instead to find the log messages produced by th= e >>>>> ignite daemon startup. >>>>> >>>>> >>>>> Any idea what's wrong? >>>>> >>>>> >>>>> Cheers, >>>>> Paolo >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Best regards, >>>> Alexei Scherbakov >>>> >>> >>> >> >> >> -- >> >> Best regards, >> Alexei Scherbakov >> > > --=20 Best regards, Alexei Scherbakov --001a114fadecd4fae5053552d66d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
This example (slightly modified) works fine for me.
https://gist.github.com/ascherbakoff/7c19da8ba568076221598d9d97e96b77<= /a>

In logs I see a problem with multicast IP = discovery.

Check if the mulitcast is enabled on yo= ur machine or better use static IP discovery[1]


2016-06-15 18:06 GMT+03:00 Paolo Di Tommaso <paolo.ditommaso@gmail.com>:
The version is 1.6.0#20160518-sha1:0b22c45b and the foll= owing is the script I'm using.=C2=A0



Cheers, p


On W= ed, Jun 15, 2016 at 5:00 PM, Alexei Scherbakov <alexey.scherbak= off@gmail.com> wrote:
I don't think it's OK.

Which I= ngite's version do you use?
<= br>
2016-06-15 15:35 GMT+03:00 Paolo Di Tommaso <= span dir=3D"ltr"><paolo.ditommaso@gmail.com>:
Great, now it works! Thanks a lot.=C2=A0

I have only a NPE during the application shutdow= n (you can find the stack trace=C2=A0at this link). Is this normal? and in any case is = there a way to avoid it?=C2=A0


Chee= rs,
Paolo



On Wed, Jun 15, 2016 = at 1:25 PM, Alexei Scherbakov <alexey.scherbakoff@gmail.com= > wrote:
Hi,
To automatically start Ignite nodes you must pass false p= arameter to 3-d IgniteContext argument like:

// ja= va
SparcContext sc =3D ...
new JavaIgniteContext<>= ;(sc, new IgniteConfigProvider(), false);;

or

// scala
SparcContext sc =3D ...
ne= w IgniteContext[String, String](sc,() =E2=87=92 configurationClo(), false)<= /div>

20= 16-06-15 13:31 GMT+03:00 Paolo Di Tommaso <paolo.ditommaso@gmail.c= om>:
Hi al= l,=C2=A0

I'm struggling deploying an Ignite applicat= ion in a Spark (local) cluster using the Embedded deploying described at this link. =C2=A0

The documentation seems suggesting that Ignite workers are automati= cally instantiated at runtime when submitting the Ignite app.
Could you please confirm that this is the expected behaviour?= =C2=A0


In my tests the when the app= lication starts it simply hangs, reporting this warning message:=C2=A0

WARN =C2=A0org.apache.ignite.spi.discovery.tcp.Tc= pDiscoverySpi =C2=A0- Failed to connect to any address from IP finder (will= retry to join topology every 2 secs): [/192.168.1.36:47500, /192.168.99.1:47500]
It looks like there are not ignite daemons to which connect to= . Also inspecting the Spark worker log I'm unable to find any message p= roduced by Ignite. I'm expecting instead to find the log messages produ= ced by the ignite daemon startup.=C2=A0


=
Any idea what's wrong?=C2=A0


Cheers,
Paolo




<= font color=3D"#888888">--

Best regards,
Alexei Scherbakov
=




--
=

Best regards,
Alexei Scherbakov




--
=

Best regards,
Alexei Scherbakov
--001a114fadecd4fae5053552d66d--