edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dale LaBossiere (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (QUARKS-173) Add load balanced parallel()
Date Mon, 09 May 2016 19:13:12 GMT

     [ https://issues.apache.org/jira/browse/QUARKS-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dale LaBossiere resolved QUARKS-173.
    Resolution: Fixed

Added PlumbingStreams.parallelBalanced(), test and recipe.

Just to capture some alternative implementation notes from the PR:

Note: there's an oplet based implementation approach to a LoadBalancedSplitter that doesn't
require "done" signaling nor separate ch "isolate()" use - one that has an internal thread
pool pulling from single tuple queue. Utilizing such a 1:n (~fanout) oplet would require adding
something like TStream.split(Split) or perhaps TStream.addOp1N(oplet-with-one-input-and-N-output).

I'm sure there's a similar functional LoadBalancedSplitter possible and maybe I'll give that
a little thought shortly. Though, at this moment there's no way for Functions to get their
hands on the Provider's thread pool nor receive explicit initialization.

> Add load balanced parallel()
> ----------------------------
>                 Key: QUARKS-173
>                 URL: https://issues.apache.org/jira/browse/QUARKS-173
>             Project: Quarks
>          Issue Type: Improvement
>            Reporter: Dale LaBossiere
>            Assignee: Dale LaBossiere
> Add an implementation that allocates a received tuple to the first "free" channel.  A
"load balanced splitter" is used internally.
> e.g.,
> ```
> public static <T,R> TStream<R> parallelBalanced(TStream<T> stream,
int width, BiFunction<TStream<T>,Integer,TStream<R>> pipeline) {
> ```

This message was sent by Atlassian JIRA

View raw message