geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephane Maldini (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GEODE-34) Introduce Reactive Streams and/or Reactor
Date Sat, 23 May 2015 13:29:17 GMT

     [ https://issues.apache.org/jira/browse/GEODE-34?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stephane Maldini updated GEODE-34:
----------------------------------
    Description: 
Current threading strategy in various places involve blocking wait, e.g:
- Put/PutAll
- Function/ResultCollector
- CacheListener handling

There is a couple small projects to help addressing this issue which would tie  very well
with Geode distributed nature. 
A first micro-library is Reactor [1] depending on Reactive Streams [2] to provide for error
isolation and latency mitigation wherever it is required. The purpose of Reactor and Reactive
Streams is to introduce a reverse non-blocking flow control to any producer:

- A component using put/putAll should not try accumulating unsafely pending writes in an async
queue such as the ones in threadPool executors if there are current put/putAll in process.
It should be actually driven by the acknowledgement from a successful write (or the acknowledgement
it has been replicated etc). The propagation of the async backpressure would tell this component
to not even try, e.g. if it is an HttpServer, stop reading incoming request, serve other requests
and resume after put has been done. This gives a predictive memory profile and a better CPU
use overall in non blocking applications, which is all the point about Reactive Streams.

- ResultCollector, Query shouldn't execute more than requested by the consumer in order to
avoid accumulating too much pending results. 

- CacheListener dispatching could benefit from implementing Reactive contract to provide for
safe concurrency model (actor-like), error propagation and like in the 2 previous examples,
async flow control that avoids pending event to accumulate somewhere in an uncontrolled fashion.

The issue here is about studying the benefits, experimenting on a branch and report any interesting
finding. Other reactive data providers can be found on the vendor side (CouchDB, MongoDB both
provide for a Reactive Streams ready query mech) and mapping side (Slick 3.0 is a pure Reactive
Streams implementation for data queries).

[1] http://projectreactor.io/docs/reference/
[2] http://reactive-streams.org

  was:
Current threading strategy in various places involve blocking wait, e.g:
- Put/PutAll
- Function/ResultCollector
- CacheListener handling

There is a couple small projects to help addressing this issue which would tie  very well
with Geode distributed nature. 
A first micro-library is Reactor [1] depending on Reactive Streams [2] to provide for error
isolation and latency mitigation wherever it is required. The purpose of Reactor and Reactive
Streams is to introduce a reverse non-blocking flow control to any producer:

- A component using put/putAll should not try accumulating unsafely pending writes in an async
queue such as the ones in threadPool executors if there are current put/putAll in process.
It should be actually driven by the acknowledgement from a successful write (or the acknowledgement
it has been replicated etc). The propagation of the async backpressure would tell this component
to not even try, e.g. if it is an HttpServer, stop reading incoming request, server other
requests and resume after put has been done. This gives a predictive memory profile and a
better CPU use overall in non blocking applications, which is all the point about Reactive
Streams.

- ResultCollector, Query shouldn't execute more than requested by the consumer in order to
avoid accumulating too much pending results. 

- CacheListener dispatching could benefit from implementing Reactive contract to provide for
safe concurrency model (actor-like), error propagation and like in the 2 previous examples,
async flow control that avoids pending event to accumulate somewhere in an uncontrolled fashion.

The issue here is about studying the benefits, experimenting on a branch and report any interesting
finding. Other reactive data providers can be found on the vendor side (CouchDB, MongoDB both
provide for a Reactive Streams ready query mech) and mapping side (Slick 3.0 is a pure Reactive
Streams implementation for data queries).

[1] http://projectreactor.io/docs/reference/
[2] http://reactive-streams.org


> Introduce Reactive Streams and/or Reactor
> -----------------------------------------
>
>                 Key: GEODE-34
>                 URL: https://issues.apache.org/jira/browse/GEODE-34
>             Project: Geode
>          Issue Type: Improvement
>          Components: core, querying
>    Affects Versions: 1.0.0-incubating
>            Reporter: Stephane Maldini
>
> Current threading strategy in various places involve blocking wait, e.g:
> - Put/PutAll
> - Function/ResultCollector
> - CacheListener handling
> There is a couple small projects to help addressing this issue which would tie  very
well with Geode distributed nature. 
> A first micro-library is Reactor [1] depending on Reactive Streams [2] to provide for
error isolation and latency mitigation wherever it is required. The purpose of Reactor and
Reactive Streams is to introduce a reverse non-blocking flow control to any producer:
> - A component using put/putAll should not try accumulating unsafely pending writes in
an async queue such as the ones in threadPool executors if there are current put/putAll in
process. It should be actually driven by the acknowledgement from a successful write (or the
acknowledgement it has been replicated etc). The propagation of the async backpressure would
tell this component to not even try, e.g. if it is an HttpServer, stop reading incoming request,
serve other requests and resume after put has been done. This gives a predictive memory profile
and a better CPU use overall in non blocking applications, which is all the point about Reactive
Streams.
> - ResultCollector, Query shouldn't execute more than requested by the consumer in order
to avoid accumulating too much pending results. 
> - CacheListener dispatching could benefit from implementing Reactive contract to provide
for safe concurrency model (actor-like), error propagation and like in the 2 previous examples,
async flow control that avoids pending event to accumulate somewhere in an uncontrolled fashion.
> The issue here is about studying the benefits, experimenting on a branch and report any
interesting finding. Other reactive data providers can be found on the vendor side (CouchDB,
MongoDB both provide for a Reactive Streams ready query mech) and mapping side (Slick 3.0
is a pure Reactive Streams implementation for data queries).
> [1] http://projectreactor.io/docs/reference/
> [2] http://reactive-streams.org



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message