spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Yuanjian (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-22565) Session-based windowing
Date Fri, 28 Sep 2018 03:24:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-22565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631298#comment-16631298
] 

Li Yuanjian edited comment on SPARK-22565 at 9/28/18 3:23 AM:
--------------------------------------------------------------

Thanks for reporting this. Actually we also met this problem in our usage, we have an implement
about session window in internal folk to resolve this. After steady running online for real
product env, we want to contribute to community within the next few days. We implemented this
by a build-in function named session_window and corresponding support for window merge in
Structure Streaming. The usage of dataframe api and SQL can be quickly browsing by the test:
 !screenshot-1.png!


was (Author: xuanyuan):
Thanks for reporting this. Actually we also met this problem in our usage, we have an implement
about session window in internal folk to resolve this. After steady running online for real
product env, we want to contribute to community within the next few days. We implemented this
by a build-in function named session_window. The usage of dataframe api and SQL can be quickly
browsing by the test:
 !screenshot-1.png!

> Session-based windowing
> -----------------------
>
>                 Key: SPARK-22565
>                 URL: https://issues.apache.org/jira/browse/SPARK-22565
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 2.2.0
>            Reporter: Richard Xin
>            Priority: Major
>         Attachments: screenshot-1.png
>
>
> I came across a requirement to support session-based windowing. for example, user activity
comes in from kafka, we want to create window per user session (if the time gap of activity
from the same user exceeds the predefined value, a new window will be created).
> I noticed that Flink does support this kind of support, any plan/schedule for spark for
this? 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message