spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Does partition by and order by works only in stateful case?
Date Fri, 13 Apr 2018 02:34:41 GMT
Hi All,

Does partition by and order by works only in stateful case?

For example:

select row_number() over (partition by id order by timestamp) from table

gives me

*SEVERE: Exception occured while submitting the query:
java.lang.RuntimeException: org.apache.spark.sql.AnalysisException:
Non-time-based windows are not supported on streaming DataFrames/Datasets;;*

I wonder what time based window means? is it not the window from over()
clause or does it mean group by(window('timestamp'), '10 minutes') like the
stateful case?

Thanks

Mime
View raw message