Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
Date: Mon, 4 Dec 2017 12:42:47 -0700 (MST)
From: Chris Berry <chriswberry@gmail.com>
To: user@ignite.apache.org
Message-ID: <1512416567989-0.post@n6.nabble.com>
In-Reply-To: <CAMP9FTSDEH_+_YKrW7oUBr8rsHuSBuGqK5FMyorLr7-N4rA09A@mail.gmail.com>
References: <1512369278083-0.post@n6.nabble.com> <CAMP9FTSDEH_+_YKrW7oUBr8rsHuSBuGqK5FMyorLr7-N4rA09A@mail.gmail.com>
Subject: Re: ignite.compute(ClusterGroup) is broken??
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
archived-at: Mon, 04 Dec 2017 19:42:52 -0000

Hi Denis,=20

Thank you for responding.

>>> Can you provide a short summary of what you are trying to achieve?

The problem is this.

Ignite begins sending compute traffic to a Node (i.e. ComputeJobs), before
that Node is truly ready to take traffic. Basically, as soon as Ignite
starts and is balanced on a Node, it begins receiving compute traffic.

This is wrong.=20
It means that as a Node starts up, it is slammed with traffic it is not
ready to handle yet.
So. We must reject that traffic until we are ready (throw an Exception).
Which, then, fails that ComputeJob over to another Node.
This, of course, works. But it is costly, in terms of system throughput as
Nodes come and go in AWS.

We need the ability to notify a Node that it is ready to handle compute
load.
Instead of Ignite *assuming* that the Node is ready as soon as Ignite is
started.

It is a classic chicken-and-egg problem.
We need Ignite started and rebalanced, because we need to interact w/ some
Ignite caches.
Thus, we need to start Ignite before we can finish enabling the app.
In addition, many other things at startup things happen in the background,
such as Kafka streaming in data to some caches, etc.
And when all of this is complete, we toggle an =E2=80=9CAlive switch=E2=80=
=9D that indicates
that the Node is ready for business.

>>> The main thing that I don't understand is why you need a=C2=A0map=C2=A0=
method?
>>> This method is supposed to be used as a part of a map-reduce pipeline.
>>> It is aimed to split one big task into many small jobs.

This is exactly how we are using it.

But the question at hand here is about whether;
ignite.cluster.compute(ClusterGroup) is broken or not.

Because my logs clearly show me that even though I am absolutely certain
that a ClusterGroup, passed to the compute() method, does NOT contain a
particular Node.
I can see that Node receiving ComputeJobs.

I do not see how that can be possible.

Thanks,=20
-- Chris=20


--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/