spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-24161) Enable debug package feature on structured streaming
Date Thu, 03 May 2018 02:40:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-24161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-24161:
------------------------------------

    Assignee: Apache Spark

> Enable debug package feature on structured streaming
> ----------------------------------------------------
>
>                 Key: SPARK-24161
>                 URL: https://issues.apache.org/jira/browse/SPARK-24161
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 2.3.0
>            Reporter: Jungtaek Lim
>            Assignee: Apache Spark
>            Priority: Major
>
> Currently, debug package has a implicit class which matches Dataset to provide debug
features on Dataset class. It doesn't work with structured streaming: it requires query is
already started, and the information can be retrieved from StreamingQuery, not Dataset. For
the same reason, "explain" had to be placed to StreamingQuery whereas it exists on Dataset.
> This issue tracks effort to enable debug package feature on structured streaming. Unlike
batch, it may have some restrictions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message