Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 31 Aug 2016 20:56:20 +0000 (UTC)
From: "Robert Conrad (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12971178.1463670988000.462208.1472676980772@Atlassian.JIRA>
In-Reply-To: <JIRA.12971178.1463670988000@Atlassian.JIRA>
References: <JIRA.12971178.1463670988000@Atlassian.JIRA> <JIRA.12971178.1463670988727@arcas>
Subject: [jira] [Commented] (SPARK-15406) Structured streaming support for
 consuming from Kafka
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 31 Aug 2016 20:56:22 -0000


    [ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453334#comment-15453334 ] 

Robert Conrad commented on SPARK-15406:
---------------------------------------

Not sure about others but my use-case is to consume a logical replication log from a database (e.g. produced by pg-bottledwater and written to kafka with avro) and thus maintain an always up-to-date spark-sql view of specific tables in the db without polling it.

> Structured streaming support for consuming from Kafka
> -----------------------------------------------------
>
>                 Key: SPARK-15406
>                 URL: https://issues.apache.org/jira/browse/SPARK-15406
>             Project: Spark
>          Issue Type: New Feature
>            Reporter: Cody Koeninger
>
> Structured streaming doesn't have support for kafka yet.  I personally feel like time based indexing would make for a much better interface, but it's been pushed back to kafka 0.10.1
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org