kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (KAFKA-4125) Provide low-level Processor API meta data in DSL layer
Date Thu, 06 Jul 2017 23:21:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthias J. Sax reassigned KAFKA-4125:
--------------------------------------

    Assignee: Jeyhun Karimov  (was: Bill Bejeck)

> Provide low-level Processor API meta data in DSL layer
> ------------------------------------------------------
>
>                 Key: KAFKA-4125
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4125
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Jeyhun Karimov
>            Priority: Minor
>              Labels: kip
>             Fix For: 0.11.1.0
>
>
> For Processor API, user can get meta data like record offset, timestamp etc via the provided
{{Context}} object. It might be useful to allow uses to access this information in DSL layer,
too.
> The idea would be, to do it "the Flink way", ie, by providing
> RichFunctions; {{mapValue()}} for example.
> Is takes a {{ValueMapper<V1, V2>}} that only has method
> {noformat}
> V2 apply(V1 value);
> {noformat}
> Thus, you cannot get any meta data within apply (it's completely "blind").
> We would add two more interfaces: {{RichFunction}} with a method
> {{open(Context context)}} and
> {noformat}
> RichValueMapper<V1, V2> extends ValueMapper<V1, V2>, RichFunction
> {noformat}
> This way, the user can chose to implement Rich- or Standard-function and
> we do not need to change existing APIs. Both can be handed into
> {{KStream.mapValues()}} for example. Internally, we check if a Rich
> function is provided, and if yes, hand in the {{Context}} object once, to
> make it available to the user who can now access it within {{apply()}} -- or
> course, the user must set a member variable in {{open()}} to hold the
> reference to the Context object.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message