kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4217) KStream.transform equivalent of flatMap
Date Thu, 24 Aug 2017 21:03:01 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140694#comment-16140694

Matthias J. Sax commented on KAFKA-4217:

[~minfing] I guess this is TBD and should be part of a KIP discussion.

However, I am not sure if we should add something like this in the first place. The current
JavaDocs of {{Transformer}} do explain how to use it already:

     * Transform the record with the given key and value.
     * Additionally, any {@link StateStore state} that is {@link KStream#transform(TransformerSupplier,
     * attached} to this operator can be accessed and modified
     * arbitrarily (cf. {@link ProcessorContext#getStateStore(String)}).
     * <p>
     * If more than one output record should be forwarded downstream {@link ProcessorContext#forward(Object,
     * {@link ProcessorContext#forward(Object, Object, int)}, and
     * {@link ProcessorContext#forward(Object, Object, String)} can be used.
     * If not record should be forwarded downstream, {@code transform} can return {@code null}.
     * @param key the key for the record
     * @param value the value for the record
     * @return new {@link KeyValue} pair&mdash;if {@code null} no key-value pair will
     * be forwarded to down stream
    R transform(final K key, final V value);

IMHO, we can just close this issue as "not an issue". [~elevy] [~guozhang] WDYT?

> KStream.transform equivalent of flatMap
> ---------------------------------------
>                 Key: KAFKA-4217
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4217
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions:
>            Reporter: Elias Levy
>              Labels: api, needs-kip, newbie
> {{KStream.transform}} gives you access to state stores while allowing you to return zero
or one transformed {{KeyValue}}.  Alas, it is unclear what method you should use if you want
to access state stores and return zero or multiple {{KeyValue}}.  Presumably you can use {{transform}},
always return {{null}}, and use {{ProcessorContext.forward}} to emit {{KeyValues}}.
> It may be good to introduce a {{transform}}-like {{flatMap}} equivalent, or allow store
access from other {{KStream}} methods, such as {{flatMap}} itself.

This message was sent by Atlassian JIRA

View raw message