phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Mahonin (JIRA)" <>
Subject [jira] [Updated] (PHOENIX-2567) phoenix-spark: DataFrame API should handle 'DATE' columns
Date Tue, 05 Jan 2016 15:00:46 GMT


Josh Mahonin updated PHOENIX-2567:
    Attachment: PHOENIX-2567.patch

> phoenix-spark: DataFrame API should handle 'DATE' columns
> ---------------------------------------------------------
>                 Key: PHOENIX-2567
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.7.0
>            Reporter: Josh Mahonin
>            Assignee: Josh Mahonin
>             Fix For: 4.7.0
>         Attachments: PHOENIX-2567.patch
> The current implementation had the 'DATE' datatype bound to a Spark SQL 'TimestampType',
which causes a casting error trying to convert from java.sql.Date to java.sql.Timestamp when
using the DataFrame API with Phoenix DATE columns.
> This patch modifies the schema handling to treat DATE columns as the Spark 'DateType'
instead. Note that Spark *drops* the hour, minute and second values from these when interfacing
using DataFrames. This follows the java.sql.Date spec, but might not useful to folks who rely
on the hour/minute/second fields working using the DataFrame API and DATE columns. A future
improvement would be to force these to be TimestampTypes instead to preserve information,
but it's less intuitive and probably shouldn't be the default behaviour.

This message was sent by Atlassian JIRA

View raw message