spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Burak Yavuz (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-21463) Output of StructuredStreaming tables don't respect user specified schema when reading back the table
Date Tue, 18 Jul 2017 20:49:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-21463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Burak Yavuz reassigned SPARK-21463:
-----------------------------------

    Assignee: Burak Yavuz

> Output of StructuredStreaming tables don't respect user specified schema when reading
back the table
> ----------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-21463
>                 URL: https://issues.apache.org/jira/browse/SPARK-21463
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL, Structured Streaming
>    Affects Versions: 2.2.0
>            Reporter: Burak Yavuz
>            Assignee: Burak Yavuz
>
> When using the MetadataLogFileIndex to read back a table, we don't respect the user provided
schema as the proper column types. This can lead to issues when trying to read strings that
look like dates that get truncated to DateType, or longs being truncated to IntegerType, just
because a long value doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message