geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephane Maldini (JIRA)" <>
Subject [jira] [Created] (GEODE-34) Introduce Reactive Streams and/or Reactor
Date Sat, 23 May 2015 13:26:17 GMT
Stephane Maldini created GEODE-34:

             Summary: Introduce Reactive Streams and/or Reactor
                 Key: GEODE-34
             Project: Geode
          Issue Type: Improvement
          Components: core, querying
    Affects Versions: 1.0.0-incubating
            Reporter: Stephane Maldini

Current threading strategy in various places involve blocking wait, e.g:
- Put/PutAll
- Function/ResultCollector
- CacheListener handling

There is a couple small project to help addressing this issue which would couple very well
with Geode distributed nature. 
A first micro-library is Reactor [1] depending on Reactive Streams [2] to provide for error
isolation and latency mitigation wherever it is required. The purpose of Reactor and Reactive
Streams is to introduce a reverse non-blocking flow control to any producer:

- A component using put/putAll should not try accumulating unsafely pending writes in an async
queue such as the ones in threadPool executors if there are current put/putAll in process.
It should be actually driven by the acknowledgement the write has been sent (or the acknowledgement
it has been replicated etc). The propagation of the async backpressure would tell this component
to not even try, e.g. if it is an HttpServer, stop reading incoming request, server other
requests and resume after put has been done. This gives a predictive memory profile and a
better CPU use overall in non blocking applications, which is all the point about Reactive

- ResultCollector, Query shouldn't execute more than requested by the consumer in order to
avoid accumulating too much pending results. 

- CacheListener dispatching could benefit from implementing Reactive contract to provide for
safe concurrency model (actor-like), error propagation and like in the 2 previous examples,
async flow control that avoids pending event to accumulate somewhere in an uncontrolled fashion.

The issue here is about studying the benefits, experimenting on a branch and report any interesting
finding. Other reactive data providers can be found on the vendor side (CouchDB, MongoDB both
provide for a Reactive Streams ready query mech) and mapping side (Slick 3.0 is a pure Reactive
Streams implementation for data queries).


This message was sent by Atlassian JIRA

View raw message