beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingsong Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-25) Add user-ready API for interacting with state
Date Fri, 20 Jan 2017 08:13:26 GMT

    [ https://issues.apache.org/jira/browse/BEAM-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831368#comment-15831368
] 

Jingsong Lee commented on BEAM-25:
----------------------------------

Are there plans to support StatefulCombineFn?
eg: select itemId, count(distinct userId) from accessTable group by ceil(timestamp to day),
itemId.
Use SetState to distinct users in CombineFn instead of use Distinct transform(bring another
Combine.perKey, not efficient).

> Add user-ready API for interacting with state
> ---------------------------------------------
>
>                 Key: BEAM-25
>                 URL: https://issues.apache.org/jira/browse/BEAM-25
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>              Labels: State
>
> Our current state API is targeted at runner implementers, not pipeline authors. As such
it has many capabilities that are not necessary nor desirable for simple use cases of stateful
ParDo (such as dynamic state tag creation). Implement a simple state intended for user access.
> (Details of our current thoughts in forthcoming design doc)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message