Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E1E4D200BC5 for ; Tue, 22 Nov 2016 23:17:59 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E0902160B1E; Tue, 22 Nov 2016 22:17:59 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5BF7C160AF1 for ; Tue, 22 Nov 2016 23:17:59 +0100 (CET) Received: (qmail 24780 invoked by uid 500); 22 Nov 2016 22:17:58 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 24755 invoked by uid 99); 22 Nov 2016 22:17:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Nov 2016 22:17:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 7B6E72C4C70 for ; Tue, 22 Nov 2016 22:17:58 +0000 (UTC) Date: Tue, 22 Nov 2016 22:17:58 +0000 (UTC) From: "Tathagata Das (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (SPARK-18373) Make KafkaSource's failOnDataLoss=false work with Spark jobs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 22 Nov 2016 22:18:00 -0000 [ https://issues.apache.org/jira/browse/SPARK-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-18373. ----------------------------------- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15820 [https://github.com/apache/spark/pull/15820] > Make KafkaSource's failOnDataLoss=false work with Spark jobs > ------------------------------------------------------------ > > Key: SPARK-18373 > URL: https://issues.apache.org/jira/browse/SPARK-18373 > Project: Spark > Issue Type: Bug > Components: Structured Streaming > Reporter: Shixiong Zhu > Assignee: Shixiong Zhu > Fix For: 2.1.0 > > > Right now failOnDataLoss=false doesn't affect Spark jobs launched by KafkaSource. The job may still fail the query when some topics are deleted or some data is aged out. We should handle these corner cases in Spark jobs as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org