spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-24741) Have a built-in AVRO data source implementation
Date Fri, 13 Jul 2018 15:56:01 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543372#comment-16543372
] 

Xiao Li commented on SPARK-24741:
---------------------------------

Please follow the umbrella JIRA: https://issues.apache.org/jira/browse/SPARK-24768. There
are multiple tasks that are related to your scenario. 

> Have a built-in AVRO data source implementation
> -----------------------------------------------
>
>                 Key: SPARK-24741
>                 URL: https://issues.apache.org/jira/browse/SPARK-24741
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL, Structured Streaming
>    Affects Versions: 2.4.0
>            Reporter: Xiao Li
>            Assignee: Gengliang Wang
>            Priority: Major
>
> Apache Avro (https://avro.apache.org) is a data serialization format. It is widely used
in the Spark and Hadoop ecosystem, especially for Kafka-based data pipelines. Using Spark-Avro
package (https://github.com/databricks/spark-avro), Spark SQL can read and write the avro
data. Making spark-Avro built-in can provide a better experience for first-time users of Spark
SQL and structured streaming. We expect the built-in Avro data source can further improve
the adoption of structured streaming. We should consider inlining https://github.com/databricks/spark-avro.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message