ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Service grid redesign
Date Sat, 28 Jul 2018 00:51:10 GMT
Anton, clients could be remote, not local. However, even if I agreed with
you, we cannot remove the current functionality from the product.

As suggested, by default services are deployed only on server nodes. If a
user wants to involve client nodes, then it should be done by specifying a
node filter. This is how the system works today. Let's not change it for no
good reason.

D.

On Fri, Jul 27, 2018 at 3:31 AM, Anton Vinogradov <av@apache.org> wrote:

> Denis,
>
> Main features of Ignite Cache are availability and throughput.
>
> I'd like to refactor Serviсe Grid to be the same.
> Main feature should be guarantee of availability and throughput (instance
> count).
>
> Client should ask grid to execute the service, that's all. No matter how
> grid will do this.
> This should be like cache 'put' or 'get' call.
>
> In case you want to execute something locally you should just implement it
> inside your application rather than deploy it to Ignite Cluster.
> There are absolutely no reason to mix local services and Service Grid.
>
> P.s. As for me, all "local" features should be deprecated, since we're
> distributed.
>
> чт, 26 июл. 2018 г. в 21:13, Denis Mekhanikov <dmekhanikov@gmail.com>:
>
> > Anton,
> >
> > I believe, there are cases, when people want to have node singleton
> > services, that are deployed to clients, as well as to all other nodes.
> > And currently clients can execute compute jobs, issued by other clients,
> > and services are not very different from them.
> > Clients may store data and run code. We shouldn't consider them as
> > "end-user nodes". Only thin clients should be run by end users.
> >
> > But I agree, that we shouldn't encourage people to use services this way.
> > So, if it doesn't complicate the implementation too much, then a warning
> in
> > log will be enough, I think.
> >
> > Denis
> >
> > чт, 26 июл. 2018 г. в 19:56, Anton Vinogradov <av@apache.org>:
> >
> > > Folks,
> > >
> > > I don't think that it's a good idea to host services on client nodes.
> > > Client topology is not stable enough, and I don't see how to guarantee
> > > availability of such services.
> > > We'll have huge problems to guarantee availability in case of blinking
> > > clients.
> > >
> > > Also, taking into account that ignite cluster can have more that one
> user
> > > it looks odd that one user able to start service at another user's
> > hardware
> > > (bitcoin miners can be disagree with me).
> > >
> > > In case you want to use nodes only to host services - all you need is
> to
> > > filter them from cache affinity functions.
> > >
> > > I propose to implement Services pretty close to Cache implementation.
> > > It's a bad idea to reinvent the weel there.
> > > Let's just analyse Cache's code and do the same for services with same
> > > guarantee.
> > >
> > > ср, 25 июл. 2018 г. в 21:58, Vyacheslav Daradur <daradurvs@gmail.com>:
> > >
> > > > Denis, long service initialization isn't a big problem for us.
> > > >
> > > > The problem is hung initialization, that means the service deployment
> > > > will never complete.
> > > > On Wed, Jul 25, 2018 at 8:08 PM Denis Mekhanikov <
> > dmekhanikov@gmail.com>
> > > > wrote:
> > > > >
> > > > > Vyacheslav,
> > > > >
> > > > > I think, that this timeout shouldn't be mandatory, and it should
be
> > > > > disabled by default.
> > > > > We should be ready for long service initialization. So, it
> shouldn't
> > be
> > > > > done in any crucial threads like discovery or exchange.
> > > > >
> > > > > Denis
> > > > >
> > > > > ср, 25 июл. 2018 г. в 15:59, Vyacheslav Daradur <
> daradurvs@gmail.com
> > >:
> > > > >
> > > > > > FYI, I've filled the tickets:
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9075
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9076
> > > > > > On Wed, Jul 25, 2018 at 12:54 PM Vyacheslav Daradur <
> > > > daradurvs@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > I think such timeout should be determined on per-service
> level.
> > > > Can we
> > > > > > make
> > > > > > > > it part of the service configuration, or pass it into
deploy
> > > > method?
> > > > > > >
> > > > > > > Agree, per ServiceConfiguration level is more flexible
> solution.
> > > > > > >
> > > > > > > > This is a great question. Will the service be able
to
> continue
> > > > > > operating
> > > > > > > > after the cache is destroyed? If not, I would undeploy
it
> > > > > > automatically. If
> > > > > > > > yes, I would keep it. Please make sure that you are
carefully
> > > > printing
> > > > > > out
> > > > > > > > informative logs in either case, to make sure that
there is
> no
> > > > magic
> > > > > > > > happening that is hidden from users.
> > > > > > >
> > > > > > > A service will be able to work till topology's change after
> that
> > we
> > > > > > > have to recalculate assignments and at this moment we won't
> > > determine
> > > > > > > suitable nodes.
> > > > > > >
> > > > > > > I will fill new tickets to work on these questions and
to
> > implement
> > > > > > > solutions in the second iteration if nobody doesn't mind.
> > > > > > > Anyway, it will have been done to a release.
> > > > > > > On Wed, Jul 25, 2018 at 12:08 PM Dmitriy Setrakyan
> > > > > > > <dsetrakyan@apache.org> wrote:
> > > > > > > >
> > > > > > > > On Tue, Jul 24, 2018 at 9:14 PM, Vyacheslav Daradur
<
> > > > > > daradurvs@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters, please help me to clarify the following
> questions:
> > > > > > > > >
> > > > > > > > > 1). According to the issue [1] we should propagate
services
> > > > > > deployment
> > > > > > > > > results to an initiator, that means we should
wait for wor
> > > > > > > > > Service#init method completion.
> > > > > > > > > How should we handle Service#init method hangup?
> > > > > > > > > I propose to introduce some kind of
> > > > > > > > > IgniteSystemProperties#IGNITE_SERVICE_INIT_TIMEOUT
to
> > interrupt
> > > > long
> > > > > > > > > initialization.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I think such timeout should be determined on per-service
> level.
> > > > Can we
> > > > > > make
> > > > > > > > it part of the service configuration, or pass it into
deploy
> > > > method?
> > > > > > > >
> > > > > > > >
> > > > > > > > > 2) Should we automatically undeploy services,
which had
> been
> > > > deployed
> > > > > > > > > using #deployKeyAffinitySingleton, on destroying
of related
> > > > > > IgniteCache?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > This is a great question. Will the service be able
to
> continue
> > > > > > operating
> > > > > > > > after the cache is destroyed? If not, I would undeploy
it
> > > > > > automatically. If
> > > > > > > > yes, I would keep it. Please make sure that you are
carefully
> > > > printing
> > > > > > out
> > > > > > > > informative logs in either case, to make sure that
there is
> no
> > > > magic
> > > > > > > > happening that is hidden from users.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Thoughts?
> > > > > > > > >
> > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3392
> > > > > > > > > On Tue, Jul 24, 2018 at 3:01 PM Vyacheslav Daradur
<
> > > > > > daradurvs@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Got it.
> > > > > > > > > >
> > > > > > > > > > Yes, we will preserve this behavior.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > > On Tue, Jul 24, 2018 at 2:20 PM Dmitriy
Setrakyan <
> > > > > > dsetrakyan@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > By default the client nodes should
be excluded form
> > service
> > > > > > > > > deployment. The
> > > > > > > > > > > only way to include clients is to explicitly
specify
> them
> > > > > > through node
> > > > > > > > > > > filter. This is how services are deployed
today and we
> > > should
> > > > > > preserve
> > > > > > > > > this
> > > > > > > > > > > behavior.
> > > > > > > > > > >
> > > > > > > > > > > D.
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Jul 24, 2018 at 11:20 AM, Denis
Mekhanikov <
> > > > > > > > > dmekhanikov@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Maybe let's keep the functionality
the way it is,
> since
> > > it
> > > > > > doesn't
> > > > > > > > > > > > interfere with the IEP?
> > > > > > > > > > > >
> > > > > > > > > > > > But I think, it's worth mentioning
as a warning in
> log,
> > > > that a
> > > > > > > > > service is
> > > > > > > > > > > > deployed on a client node.
> > > > > > > > > > > >
> > > > > > > > > > > > Denis
> > > > > > > > > > > >
> > > > > > > > > > > > вт, 24 июл. 2018 г. в 12:44,
Vyacheslav Daradur <
> > > > > > daradurvs@gmail.com
> > > > > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > No, it's doesn't complicate
implementation on the
> > > current
> > > > > > stage.
> > > > > > > > > > > > >
> > > > > > > > > > > > > But we will have to change
assignment function to
> > > forbid
> > > > > > client
> > > > > > > > > nodes
> > > > > > > > > > > > > even if configuration's node
filter resolves them
> it
> > > can
> > > > be
> > > > > > not
> > > > > > > > > > > > > transparent for the end user.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think the only use case
to have such behavior is:
> > > > hosting
> > > > > > of not
> > > > > > > > > > > > > collocated services on data
free nodes with access
> to
> > > > > > IgniteCaches
> > > > > > > > > on
> > > > > > > > > > > > > remote nodes in the same
cluster.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Jul 24, 2018 at 12:10
PM Denis Mekhanikov <
> > > > > > > > > dmekhanikov@gmail.com
> > > > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I don't think, that
client nodes, that host
> > services
> > > > make
> > > > > > much
> > > > > > > > > sense.
> > > > > > > > > > > > > > May we forbid it? Does
anybody know, when it may
> be
> > > > useful?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Vyacheslav, does it
complicate the implementation
> > > > somehow?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > вт, 24 июл. 2018
г. в 11:57, Vyacheslav Daradur <
> > > > > > > > > daradurvs@gmail.com>:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi, Igniters!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I am close to completing
the main tasks and I'm
> > > > going to
> > > > > > > > > request a
> > > > > > > > > > > > > > > review in a couple
weeks.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I have a question
about the new design:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The current implementation
of Service Grid
> > doesn't'
> > > > take
> > > > > > into
> > > > > > > > > account
> > > > > > > > > > > > > > > Ignition#client(true).
> > > > > > > > > > > > > > > It means that *clients*
nodes are able to host
> > > > services.
> > > > > > There
> > > > > > > > > are
> > > > > > > > > > > > > > > some tests that
expect such behavior.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Services assignments
are managed by a predicate
> > > only
> > > > > > > > > > > > > > >
> > > (ServiceConfiguration#setNodeFilter(IgnitePredicate<
> > > > > > > > > ClusterNode>).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Should deployment
on clients nodes be forbidden
> > or
> > > we
> > > > > > > > > shouldn't mix
> > > > > > > > > > > > > > > concepts for IgniteCache
with Service Grid?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, Jun 20,
2018 at 1:46 AM Dmitriy
> > Setrakyan <
> > > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Tue, Jun
19, 2018 at 1:50 PM, Vyacheslav
> > > > Daradur <
> > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi Dmitriy,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Yes,
the task [1] is planned to be
> > implemented
> > > > once
> > > > > > the
> > > > > > > > > main
> > > > > > > > > > > > tasks
> > > > > > > > > > > > > > > > > will
be completed.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > [1]
> > > > > > https://issues.apache.org/jira/browse/IGNITE-8367
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Awesome! This
is a huge addition to the
> > project.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Tue,
Jun 19, 2018 at 10:06 PM Dmitriy
> > > > Setrakyan
> > > > > > > > > > > > > > > > > <dsetrakyan@apache.org>
wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Hi Vyacheslav,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
How about service redeployment in case if
> > > user
> > > > > > wants to
> > > > > > > > > update
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > code?
> > > > > > > > > > > > > > > > > Is
> > > > > > > > > > > > > > > > > >
this planned?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
D.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > Best Regards, Vyacheslav
D.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Best Regards, Vyacheslav
D.
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Vyacheslav D.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message