ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: Continuations for services
Date Tue, 28 Mar 2017 18:53:18 GMT
Vova,

Take a look at GridJobProcessor#processJobExecuteRequest method. In
particular, I'm talking about this piece of code:

if (ctx.localNodeId().equals(node.id())) {
    // Always execute in another thread for local node.
    executeAsync(job);

    // No sync execution.
    job = null;
}

executeAsync() method simply submits the job to public pool. This happens
every time job is executed locally and I'm having hard time understanding
why. In my view it negatively affects performance as we do context switch,
and creates yet another condition for starvation. If user executes it
synchronously, it doesn't make sense.

BTW, we don't do the same for services. If a service proxy is created for a
service which is available locally, we just return this instance. When user
invokes a method, it will be executed synchronously in the same thread.

-Val

On Tue, Mar 28, 2017 at 12:59 AM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Valya,
>
> Could you please point me where we delegate thread to another pool? I
> remember we did something around this when working with some streamer
> problems.
>
> On Tue, Mar 28, 2017 at 4:18 AM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
> > Vova,
> >
> > Agree, I already merged IGNITE-4802.
> >
> > However, I'm still interested why we always switch to another thread when
> > executing a job locally. Does anyone have an idea why we do this?
> >
> > -Val
> >
> > On Sat, Mar 25, 2017 at 7:22 AM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > Valya,
> > >
> > > I don't think it will resolve all the cases. For example, what if I
> > execute
> > > remote job from another job? "Remoteness" could be caused by job natire
> > > (broadcast), specific cluster group, or affinity call/run on remote
> key.
> > > Also starvation is possible not only on local node, but between nodes,
> > and
> > > in this case separate pool is the only reliable solution.
> > >
> > > On Sat, Mar 25, 2017 at 1:50 AM, Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com> wrote:
> > >
> > > > Folks,
> > > >
> > > > I tend to think that a separate pool for services is not right
> > solution.
> > > >
> > > > We currently execute any new compute job in a separate thread, even
> if
> > > > we're already in the public pool (see code in
> > > > GridJobProcessor#processJobExecuteRequest). What is the reason for
> > this?
> > > > When a job synchronously executes another job on the same node, can
> we
> > > just
> > > > execute it in the same thread? This seems to fix all starvation
> issues
> > > > discussed in this thread.
> > > >
> > > > Am I missing something?
> > > >
> > > > -Val
> > > >
> > > > On Thu, Mar 9, 2017 at 2:32 AM, Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com> wrote:
> > > >
> > > > > OK, I created it: https://issues.apache.org/
> jira/browse/IGNITE-4802
> > > > >
> > > > > -Val
> > > > >
> > > > > On Thu, Mar 9, 2017 at 9:58 AM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > >> Valya,
> > > > >>
> > > > >> Not yet.
> > > > >>
> > > > >> On Thu, Mar 9, 2017 at 11:50 AM, Valentin Kulichenko <
> > > > >> valentin.kulichenko@gmail.com> wrote:
> > > > >>
> > > > >> > Vladimir,
> > > > >> >
> > > > >> > This makes sense to me. Is there a ticket for separate thread
> pool
> > > for
> > > > >> > services?
> > > > >> >
> > > > >> > -Val
> > > > >> >
> > > > >> > On Thu, Mar 9, 2017 at 2:52 AM, Dmitriy Setrakyan <
> > > > >> dsetrakyan@apache.org>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Vladimr, it sounds like what you are suggesting is
allowing
> > users
> > > > >> specify
> > > > >> > > named executors in configuration and then use them
from code,
> > > > right? I
> > > > >> > > think I like this idea very much.
> > > > >> > >
> > > > >> > > On Wed, Mar 8, 2017 at 1:46 PM, Vladimir Ozerov <
> > > > vozerov@gridgain.com
> > > > >> >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Continuations is not very good idea. It is useful
if user
> has
> > > > simple
> > > > >> > > logic
> > > > >> > > > when one job calls another from within the same
> > execute/run/call
> > > > >> > method.
> > > > >> > > > But if you have complex logic with OOP abstractions
and
> > reusable
> > > > >> > > > components, nested job call can be located many
stack frames
> > > down
> > > > >> from
> > > > >> > > > parent job. In this case continuations are unusable.
> > > > >> > > >
> > > > >> > > > More convenient approach is to map separate jobs
to separate
> > > > thread
> > > > >> > > pools.
> > > > >> > > > This technique is successfully employed in Hazelcast.
You
> just
> > > > >> define
> > > > >> > > > additional executors and say that job A is to
be executed
> one
> > > > thread
> > > > >> > > pool,
> > > > >> > > > and job B on another.
> > > > >> > > >
> > > > >> > > > The same technique is applicable for services:
> > > > >> > > >
> > > > >> > > > class MyService implements Service {
> > > > >> > > >     @IgniteInstanceResource
> > > > >> > > >     Ignite ignite;
> > > > >> > > >
> > > > >> > > >     void myMethod() {
> > > > >> > > >
> > > > >> > > > ignite.service().withExecutor("myExecutor").service("
> > > > >> > > > myService").nestedCall();
> > > > >> > > >     }
> > > > >> > > > }
> > > > >> > > >
> > > > >> > > > All in all I would do the following:
> > > > >> > > > 1) Create separate built-in pool for services
to make sure
> > that
> > > in
> > > > >> > simple
> > > > >> > > > cases users are able to call compute jobs from
service
> > methods.
> > > > >> > > > 2) Implement custom executors which will be applicable
for
> > both
> > > > >> compute
> > > > >> > > [1]
> > > > >> > > > and service components.
> > > > >> > > >
> > > > >> > > > [1] https://issues.apache.org/jira/browse/IGNITE-4699
> > > > >> > > >
> > > > >> > > > 08 марта 2017 г. 23:01 пользователь
"Dmitriy Setrakyan" <
> > > > >> > > > dsetrakyan@apache.org> написал:
> > > > >> > > >
> > > > >> > > > > On Wed, Mar 8, 2017 at 11:58 AM, Valentin
Kulichenko <
> > > > >> > > > > valentin.kulichenko@gmail.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Separate thread pool will not solve
the case of calling
> a
> > > > >> service
> > > > >> > > from
> > > > >> > > > > > another service.
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > > Why not? The caller thread should block.
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message