spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-19779) structured streaming exist needless tmp file
Date Fri, 03 Mar 2017 05:21:45 GMT

     [ https://issues.apache.org/jira/browse/SPARK-19779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shixiong Zhu reassigned SPARK-19779:
------------------------------------

    Assignee: Feng Gui

> structured streaming exist needless tmp file 
> ---------------------------------------------
>
>                 Key: SPARK-19779
>                 URL: https://issues.apache.org/jira/browse/SPARK-19779
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 2.0.3, 2.1.1, 2.2.0
>            Reporter: Feng Gui
>            Assignee: Feng Gui
>            Priority: Minor
>             Fix For: 2.0.3, 2.1.1, 2.2.0
>
>
> The PR (https://github.com/apache/spark/pull/17012) can to fix restart a Structured Streaming
application using hdfs as fileSystem, but also exist a problem that a tmp file of delta file
is still reserved in hdfs. And Structured Streaming don't delete the tmp file generated when
restart streaming job in future, so we need to delete the tmp file after restart streaming
job.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message