incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthieu Morel (JIRA)" <>
Subject [jira] [Commented] (S4-62) Multithreaded Streams
Date Wed, 11 Jul 2012 09:05:33 GMT


Matthieu Morel commented on S4-62:

Matt Welsh's feedback on SEDA (
) could be taken into account. S4 follows a similar stage-driven architecture. Whereas it
might be difficult to scale out by grouping stages into a single "thread pool domain" as suggested
in the blog post, an interesting recommendation is to put "_a separate thread pool and queue
in front of a group of stages that have long latency or nondeterministic runtime, such as
performing disk I/O._" This corresponds to the proposal of this ticket. 

The platform could even provide this feature automatically when streams or PEs are identified
as I/O consumers.

> Multithreaded Streams
> ---------------------
>                 Key: S4-62
>                 URL:
>             Project: Apache S4
>          Issue Type: Improvement
>    Affects Versions: 0.5
>            Reporter: Daniel Gómez Ferro
>         Attachments: S4-62-multithreaded-streams.patch, S4-62-multithreaded-streams.patch
> Currently, each Stream has one Thread in charge of processing the incoming events on
the appropriate PE. If one PE blocks it's execution while processing an event, the whole Stream
would be blocked. The current solution is for a PE to manage his own async thread, which I
don't think it's nice.
> It would be better to have a configurable number of threads that would take care of the
execution of the incoming events.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message