kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4217) KStream.transform equivalent of flatMap
Date Thu, 24 Aug 2017 22:14:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140832#comment-16140832

Guozhang Wang commented on KAFKA-4217:

The issue with the current recommended {{forward}} call is that it looses strong typing. I.e.
if users forward a <key, value> where value type is not R it would only error-out at
runtime. Thus this JIRA was maintained as a reminder to improve on this, such that if we have
a new API added we can then updated the java doc such that {{forward}} can be used but not
recommended because of strong typing.

As for [~Yohan123]'s question, the original purpose was to add a `flatTransform` and `flatTransformValues`
functions that gives the following API:

public <K1, V1> KStream<K1, V1> flatTransform(final TransformerSupplier<? super
K, ? super V, ? extends Iterable<? extends KeyValue<? extends K1, ? extends V1>>>
transformerSupplier, ...)

public <V1> KStream<K, V1> flatTransform(final ValueTransformerSupplier<? super
V, ? extends Iterable<? extends V1>> valueTransformerSupplier, ...)

> KStream.transform equivalent of flatMap
> ---------------------------------------
>                 Key: KAFKA-4217
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4217
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions:
>            Reporter: Elias Levy
>              Labels: api, needs-kip, newbie
> {{KStream.transform}} gives you access to state stores while allowing you to return zero
or one transformed {{KeyValue}}.  Alas, it is unclear what method you should use if you want
to access state stores and return zero or multiple {{KeyValue}}.  Presumably you can use {{transform}},
always return {{null}}, and use {{ProcessorContext.forward}} to emit {{KeyValues}}.
> It may be good to introduce a {{transform}}-like {{flatMap}} equivalent, or allow store
access from other {{KStream}} methods, such as {{flatMap}} itself.

This message was sent by Atlassian JIRA

View raw message