edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dale LaBossiere (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (EDGENT-91) Support joining a stream against another stream/window
Date Tue, 23 Aug 2016 15:27:21 GMT

     [ https://issues.apache.org/jira/browse/EDGENT-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dale LaBossiere resolved EDGENT-91.
-----------------------------------
    Resolution: Fixed

Not sure when the work was done, or the PR, but TStream now supports `join()` and `joinLast()`

> Support joining a stream against another stream/window
> ------------------------------------------------------
>
>                 Key: EDGENT-91
>                 URL: https://issues.apache.org/jira/browse/EDGENT-91
>             Project: Edgent
>          Issue Type: Improvement
>            Reporter: Will Marshall
>            Assignee: Will Marshall
>
> Quarks should support the ability to join a stream against another stream or window.
The interface would look something like this:
> {code}
>     <J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K>
window, BiFunction<T, List<U>, J> joiner);
>     
>     <J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T,
U, J> joiner);
>     <J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U>
lastStream, Function<U, K> lastStreamKeyer, BiFunction<T, U, J> joiner);
> {code}
> Copied from: http://ibmstreams.github.io/streamsx.topology/doc/javadoc/index.html
> For *join(Function<T, K> keyer, TWindow<U, K> window, BiFunction<T, List<U>,
J> joiner)*:
> Join this stream with a partitioned window of type U with key type K. For each tuple
on this stream, it is joined with the contents of window for the key keyer.apply(tuple). Each
tuple is passed into joiner and the return value is submitted to the returned stream. If call
returns null then no tuple is submitted.
> For  *joinLast(Function<T, K> keyer, TStream<U> lastStream, Function<U,
K> lastStreamKeyer, BiFunction<T, U, J> joiner)*:
> Join this stream with the last tuple seen on a stream of type U with partitioning. For
each tuple on this stream, it is joined with the last tuple seen on lastStream with a matching
key (of type K). Each tuple t on this stream will match the last tuple u on lastStream if
keyer.apply(t).equals(lastStreamKeyer.apply(u)) is true. The assumption is made that the key
classes correctly implement the contract for equals and hashCode(). Each tuple is passed into
joiner and the return value is submitted to the returned stream. If call returns null then
no tuple is submitted. 
> For *joinLast(TStream<U> lastStream, BiFunction<T, U, J> joiner)*:
> Join this stream with the last tuple seen on a stream of type U. For each tuple on this
stream, it is joined with the last tuple seen on lastStream. Each tuple is passed into joiner
and the return value is submitted to the returned stream. If call returns null then no tuple
is submitted.
> This is a simplified version of join(TWindow, BiFunction) where instead the window contents
are passed as a single tuple of type U rather than a list containing one tuple. If no tuple
has been seen on lastStream then null will be passed as the second argument to joiner.
> *Issues:*
> - The definition of Oplet<I, O> requires that all input tuples be of the same type.
If streamA and streamB are of different types, it's not clear how to create a "join" oplet
where one input port has the tuples of streamA and another has the tuples of streamB.
> - The windowing library doesn't currently support accessing the contents of a partition
from a quarks.window.Window reference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message