Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAGcMBHiYuq8hFSgGRmxJ4-qkikDivtZOdZsYi29nLVqMcRS7gw@mail.gmail.com>
References: 
 <CAP3QaUHWR4eU_2Y7c5zn=M6KR56QOtAUzuHjwW+iKpBD_X8_iQ@mail.gmail.com>
	<CAGt6o8=_bySVgT1RL10o+UQRGGRWH5GZ-4aJRhD4PwBDkY57_A@mail.gmail.com>
	<CAP3QaUH28m6wTiUNSZT8fUHCd=k6yP_U6hZJ2bZ6oyZqXF6auA@mail.gmail.com>
	<CAGt6o8n-xMrXxPBMpRFqDuTrvy2O3fCKY-FTgS=Wa_+aYo+acA@mail.gmail.com>
	<SNT147-W6142B2D7B9D71E6F461A8FCF870@phx.gbl>
	<CAFx+i-ZjQYJKvVNgQDM2sR5OKwSwPsa9F+e5Ksyh40mq=ZRFMQ@mail.gmail.com>
	<CAGt6o8=6rPmSkBEtB-ocOnCvyvADcPdGaFwEosZnf_OVKjwZhQ@mail.gmail.com>
	<CAGcMBHiYuq8hFSgGRmxJ4-qkikDivtZOdZsYi29nLVqMcRS7gw@mail.gmail.com>
Date: Wed, 30 Mar 2016 11:44:35 +0300
Message-ID: 
 <CAGt6o8kbBSjf7bPN5P92ke5r9uKxW_5fb5sH-s4xOqqPTw_Ecw@mail.gmail.com>
Subject: Re: API for asynchronous execution.
From: Vladimir Ozerov <vozerov@gridgain.com>
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=001a1145a1f2cefb05052f402757

--001a1145a1f2cefb05052f402757
Content-Type: text/plain; charset=UTF-8

Does it mean that if cache update rate is greater than filter execution
rate, then at some point we will stop reading messages from socket? If yes,
then it seems we still cannot execute cache operations:
1) Filter starts cache operation for a key. Current node is backup for this
key.
2) Cache message is sent to primary node
3) Primary sends message back to current node.
4) Message is never read because of backpressure. Cache operation and
filter never complete.

Am I missing something?

On Wed, Mar 30, 2016 at 11:23 AM, Yakov Zhdanov <yzhdanov@apache.org> wrote:

> Vladimir,
>
> Communication should stop reading from connection is there are too many
> unprocessed messages. Sender will be blocked on putting message to queue.
>
> --Yakov
>
> 2016-03-30 11:11 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
>
> > Guys,
> >
> > Can you explain how backpressure control is implemented? What if event
> > arrival speed is greater than filter processing speed?
> >
> > On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov <sboikov@gridgain.com>
> > wrote:
> >
> > > Andrey,
> > >
> > > I agree that current situation with threading in Ignite is very
> > > inconvenient when user callbacks execute some non-trivial code. But
> > > changing this to async dispatch is huge refactoring, even changing this
> > > just for continuous queries callback is not so easy task.
> > >
> > > We can start with https://issues.apache.org/jira/browse/IGNITE-2004,
> and
> > > if
> > > more users complains arise we can think about changing others parts of
> > > system.
> > >
> > > For now we need decisions for these points:
> > > - how to specify that callback should be run asynchronously (Nikolay
> > > suggested marker interface IgniteAsyncCallback, or
> @IgniteAsyncCallback)
> > > - where these callbacks are executed, AFAIK Nikolay added special pool
> > > which is configured in IgniteConfiguration (something like
> > > IgniteConfiguration.asyncCallbackThreadPoolSize)
> > >
> > > Regards
> > >
> > >
> > > On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev <
> > andrewkornev@hotmail.com>
> > > wrote:
> > >
> > > > Vladimir, Igniters
> > > >
> > > > Here are my 2 cents.
> > > >
> > > > The current situation with threading when it comes to executing user
> > > > callbacks -- the CQ filters (either local or remote), the CQ
> listeners,
> > > the
> > > > event listeners, the messaging listeners, the entry processors (did I
> > > miss
> > > > anything?) -- is pretty sad. The callbacks may get executed on a
> system
> > > > pool's thread, public pool's, utility pool's, discovery worker
> thread,
> > > > application thread, to name a few. It causes a lot of grief and
> > > suffering,
> > > > hard-to-fix races, dead locks and other bugs.
> > > >
> > > > I guess it's always possible to come up with a more or less
> reasonable
> > > > explanation to such predicament (which usually boils down to "It is
> so
> > > > because this is how it's implemented"), but I, as a user, could not
> > care
> > > > less. I want consistency. I want all my callbacks (including Entry
> > > > Processors!) to be executed on the public pool's threads, to be
> > precise.
> > > > This is not the first time I complain about this, and I really think
> > it's
> > > > time to fix this mess.
> > > >
> > > > For a good example of how to implement ordered async dispatch of
> > > callbacks
> > > > on large scale, one only needs to look at Akka (or Reactor
> > > > https://github.com/reactor/reactor).  Coherence also managed to get
> it
> > > > right (in my opinion, that is).
> > > >
> > > > Regards
> > > > Andrey
> > > >
> > > >
> > >
> >
>

--001a1145a1f2cefb05052f402757--