ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anton Vinogradov ...@apache.org>
Subject Re: Service grid redesign
Date Fri, 27 Jul 2018 10:31:43 GMT
Denis,

Main features of Ignite Cache are availability and throughput.

I'd like to refactor Serviсe Grid to be the same.
Main feature should be guarantee of availability and throughput (instance
count).

Client should ask grid to execute the service, that's all. No matter how
grid will do this.
This should be like cache 'put' or 'get' call.

In case you want to execute something locally you should just implement it
inside your application rather than deploy it to Ignite Cluster.
There are absolutely no reason to mix local services and Service Grid.

P.s. As for me, all "local" features should be deprecated, since we're
distributed.

чт, 26 июл. 2018 г. в 21:13, Denis Mekhanikov <dmekhanikov@gmail.com>:

> Anton,
>
> I believe, there are cases, when people want to have node singleton
> services, that are deployed to clients, as well as to all other nodes.
> And currently clients can execute compute jobs, issued by other clients,
> and services are not very different from them.
> Clients may store data and run code. We shouldn't consider them as
> "end-user nodes". Only thin clients should be run by end users.
>
> But I agree, that we shouldn't encourage people to use services this way.
> So, if it doesn't complicate the implementation too much, then a warning in
> log will be enough, I think.
>
> Denis
>
> чт, 26 июл. 2018 г. в 19:56, Anton Vinogradov <av@apache.org>:
>
> > Folks,
> >
> > I don't think that it's a good idea to host services on client nodes.
> > Client topology is not stable enough, and I don't see how to guarantee
> > availability of such services.
> > We'll have huge problems to guarantee availability in case of blinking
> > clients.
> >
> > Also, taking into account that ignite cluster can have more that one user
> > it looks odd that one user able to start service at another user's
> hardware
> > (bitcoin miners can be disagree with me).
> >
> > In case you want to use nodes only to host services - all you need is to
> > filter them from cache affinity functions.
> >
> > I propose to implement Services pretty close to Cache implementation.
> > It's a bad idea to reinvent the weel there.
> > Let's just analyse Cache's code and do the same for services with same
> > guarantee.
> >
> > ср, 25 июл. 2018 г. в 21:58, Vyacheslav Daradur <daradurvs@gmail.com>:
> >
> > > Denis, long service initialization isn't a big problem for us.
> > >
> > > The problem is hung initialization, that means the service deployment
> > > will never complete.
> > > On Wed, Jul 25, 2018 at 8:08 PM Denis Mekhanikov <
> dmekhanikov@gmail.com>
> > > wrote:
> > > >
> > > > Vyacheslav,
> > > >
> > > > I think, that this timeout shouldn't be mandatory, and it should be
> > > > disabled by default.
> > > > We should be ready for long service initialization. So, it shouldn't
> be
> > > > done in any crucial threads like discovery or exchange.
> > > >
> > > > Denis
> > > >
> > > > ср, 25 июл. 2018 г. в 15:59, Vyacheslav Daradur <daradurvs@gmail.com
> >:
> > > >
> > > > > FYI, I've filled the tickets:
> > > > > https://issues.apache.org/jira/browse/IGNITE-9075
> > > > > https://issues.apache.org/jira/browse/IGNITE-9076
> > > > > On Wed, Jul 25, 2018 at 12:54 PM Vyacheslav Daradur <
> > > daradurvs@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > I think such timeout should be determined on per-service
level.
> > > Can we
> > > > > make
> > > > > > > it part of the service configuration, or pass it into deploy
> > > method?
> > > > > >
> > > > > > Agree, per ServiceConfiguration level is more flexible solution.
> > > > > >
> > > > > > > This is a great question. Will the service be able to continue
> > > > > operating
> > > > > > > after the cache is destroyed? If not, I would undeploy
it
> > > > > automatically. If
> > > > > > > yes, I would keep it. Please make sure that you are carefully
> > > printing
> > > > > out
> > > > > > > informative logs in either case, to make sure that there
is no
> > > magic
> > > > > > > happening that is hidden from users.
> > > > > >
> > > > > > A service will be able to work till topology's change after
that
> we
> > > > > > have to recalculate assignments and at this moment we won't
> > determine
> > > > > > suitable nodes.
> > > > > >
> > > > > > I will fill new tickets to work on these questions and to
> implement
> > > > > > solutions in the second iteration if nobody doesn't mind.
> > > > > > Anyway, it will have been done to a release.
> > > > > > On Wed, Jul 25, 2018 at 12:08 PM Dmitriy Setrakyan
> > > > > > <dsetrakyan@apache.org> wrote:
> > > > > > >
> > > > > > > On Tue, Jul 24, 2018 at 9:14 PM, Vyacheslav Daradur <
> > > > > daradurvs@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Igniters, please help me to clarify the following
questions:
> > > > > > > >
> > > > > > > > 1). According to the issue [1] we should propagate
services
> > > > > deployment
> > > > > > > > results to an initiator, that means we should wait
for wor
> > > > > > > > Service#init method completion.
> > > > > > > > How should we handle Service#init method hangup?
> > > > > > > > I propose to introduce some kind of
> > > > > > > > IgniteSystemProperties#IGNITE_SERVICE_INIT_TIMEOUT
to
> interrupt
> > > long
> > > > > > > > initialization.
> > > > > > > >
> > > > > > >
> > > > > > > I think such timeout should be determined on per-service
level.
> > > Can we
> > > > > make
> > > > > > > it part of the service configuration, or pass it into deploy
> > > method?
> > > > > > >
> > > > > > >
> > > > > > > > 2) Should we automatically undeploy services, which
had been
> > > deployed
> > > > > > > > using #deployKeyAffinitySingleton, on destroying of
related
> > > > > IgniteCache?
> > > > > > > >
> > > > > > > >
> > > > > > > This is a great question. Will the service be able to continue
> > > > > operating
> > > > > > > after the cache is destroyed? If not, I would undeploy
it
> > > > > automatically. If
> > > > > > > yes, I would keep it. Please make sure that you are carefully
> > > printing
> > > > > out
> > > > > > > informative logs in either case, to make sure that there
is no
> > > magic
> > > > > > > happening that is hidden from users.
> > > > > > >
> > > > > > >
> > > > > > > > Thoughts?
> > > > > > > >
> > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3392
> > > > > > > > On Tue, Jul 24, 2018 at 3:01 PM Vyacheslav Daradur
<
> > > > > daradurvs@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Got it.
> > > > > > > > >
> > > > > > > > > Yes, we will preserve this behavior.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > > On Tue, Jul 24, 2018 at 2:20 PM Dmitriy Setrakyan
<
> > > > > dsetrakyan@apache.org>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > By default the client nodes should be excluded
form
> service
> > > > > > > > deployment. The
> > > > > > > > > > only way to include clients is to explicitly
specify them
> > > > > through node
> > > > > > > > > > filter. This is how services are deployed
today and we
> > should
> > > > > preserve
> > > > > > > > this
> > > > > > > > > > behavior.
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > > On Tue, Jul 24, 2018 at 11:20 AM, Denis
Mekhanikov <
> > > > > > > > dmekhanikov@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Maybe let's keep the functionality
the way it is, since
> > it
> > > > > doesn't
> > > > > > > > > > > interfere with the IEP?
> > > > > > > > > > >
> > > > > > > > > > > But I think, it's worth mentioning
as a warning in log,
> > > that a
> > > > > > > > service is
> > > > > > > > > > > deployed on a client node.
> > > > > > > > > > >
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > > вт, 24 июл. 2018 г. в 12:44,
Vyacheslav Daradur <
> > > > > daradurvs@gmail.com
> > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > No, it's doesn't complicate implementation
on the
> > current
> > > > > stage.
> > > > > > > > > > > >
> > > > > > > > > > > > But we will have to change assignment
function to
> > forbid
> > > > > client
> > > > > > > > nodes
> > > > > > > > > > > > even if configuration's node filter
resolves them it
> > can
> > > be
> > > > > not
> > > > > > > > > > > > transparent for the end user.
> > > > > > > > > > > >
> > > > > > > > > > > > I think the only use case to have
such behavior is:
> > > hosting
> > > > > of not
> > > > > > > > > > > > collocated services on data free
nodes with access to
> > > > > IgniteCaches
> > > > > > > > on
> > > > > > > > > > > > remote nodes in the same cluster.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Jul 24, 2018 at 12:10
PM Denis Mekhanikov <
> > > > > > > > dmekhanikov@gmail.com
> > > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't think, that client
nodes, that host
> services
> > > make
> > > > > much
> > > > > > > > sense.
> > > > > > > > > > > > > May we forbid it? Does anybody
know, when it may be
> > > useful?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Vyacheslav, does it complicate
the implementation
> > > somehow?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Denis
> > > > > > > > > > > > >
> > > > > > > > > > > > > вт, 24 июл. 2018 г.
в 11:57, Vyacheslav Daradur <
> > > > > > > > daradurvs@gmail.com>:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi, Igniters!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I am close to completing
the main tasks and I'm
> > > going to
> > > > > > > > request a
> > > > > > > > > > > > > > review in a couple weeks.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I have a question about
the new design:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The current implementation
of Service Grid
> doesn't'
> > > take
> > > > > into
> > > > > > > > account
> > > > > > > > > > > > > > Ignition#client(true).
> > > > > > > > > > > > > > It means that *clients*
nodes are able to host
> > > services.
> > > > > There
> > > > > > > > are
> > > > > > > > > > > > > > some tests that expect
such behavior.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Services assignments
are managed by a predicate
> > only
> > > > > > > > > > > > > >
> > (ServiceConfiguration#setNodeFilter(IgnitePredicate<
> > > > > > > > ClusterNode>).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Should deployment on
clients nodes be forbidden
> or
> > we
> > > > > > > > shouldn't mix
> > > > > > > > > > > > > > concepts for IgniteCache
with Service Grid?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Jun 20, 2018
at 1:46 AM Dmitriy
> Setrakyan <
> > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Tue, Jun 19,
2018 at 1:50 PM, Vyacheslav
> > > Daradur <
> > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi Dmitriy,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Yes, the task
[1] is planned to be
> implemented
> > > once
> > > > > the
> > > > > > > > main
> > > > > > > > > > > tasks
> > > > > > > > > > > > > > > > will be completed.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > [1]
> > > > > https://issues.apache.org/jira/browse/IGNITE-8367
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Awesome! This is
a huge addition to the
> project.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Tue, Jun
19, 2018 at 10:06 PM Dmitriy
> > > Setrakyan
> > > > > > > > > > > > > > > > <dsetrakyan@apache.org>
wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi Vyacheslav,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > How about
service redeployment in case if
> > user
> > > > > wants to
> > > > > > > > update
> > > > > > > > > > > > the
> > > > > > > > > > > > > > code?
> > > > > > > > > > > > > > > > Is
> > > > > > > > > > > > > > > > > this
planned?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Best Regards, Vyacheslav
D.
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards, Vyacheslav D.
> > > > >
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message