spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source
Date Tue, 07 Aug 2018 15:39:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571852#comment-16571852
] 

Thomas Graves commented on SPARK-24924:
---------------------------------------

thanks, I missed it in the output for spark as I was just looking at table properties. 

So what you are saying is that without this change to map databricks avro to our internal
avro, the only way to update hive tables to use the internal avro version is to have them
manually set the table properties? 

Do you know off hand if you are able to write to a hive table with datasource "com.databricks.spark.avro"
using the internal avro version or does it error?  

> Add mapping for built-in Avro data source
> -----------------------------------------
>
>                 Key: SPARK-24924
>                 URL: https://issues.apache.org/jira/browse/SPARK-24924
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Dongjoon Hyun
>            Assignee: Dongjoon Hyun
>            Priority: Minor
>             Fix For: 2.4.0
>
>
> This issue aims to the followings.
>  # Like `com.databricks.spark.csv` mapping, we had better map `com.databricks.spark.avro`
to built-in Avro data source.
>  # Remove incorrect error message, `Please find an Avro package at ...`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message