spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tathagata Das (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-18373) Make KafkaSource's failOnDataLoss=false work with Spark jobs
Date Tue, 22 Nov 2016 22:17:58 GMT

     [ https://issues.apache.org/jira/browse/SPARK-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tathagata Das resolved SPARK-18373.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: 2.1.0

Issue resolved by pull request 15820
[https://github.com/apache/spark/pull/15820]

> Make KafkaSource's failOnDataLoss=false work with Spark jobs
> ------------------------------------------------------------
>
>                 Key: SPARK-18373
>                 URL: https://issues.apache.org/jira/browse/SPARK-18373
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>            Reporter: Shixiong Zhu
>            Assignee: Shixiong Zhu
>             Fix For: 2.1.0
>
>
> Right now failOnDataLoss=false doesn't affect Spark jobs launched by KafkaSource. The
job may still fail the query when some topics are deleted or some data is aged out. We should
handle these corner cases in Spark jobs as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message