phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Mahonin (JIRA)" <>
Subject [jira] [Created] (PHOENIX-2567) phoenix-spark: DataFrame API should handle 'DATE' columns
Date Tue, 05 Jan 2016 14:58:39 GMT
Josh Mahonin created PHOENIX-2567:

             Summary: phoenix-spark: DataFrame API should handle 'DATE' columns
                 Key: PHOENIX-2567
             Project: Phoenix
          Issue Type: Bug
    Affects Versions: 4.7.0
            Reporter: Josh Mahonin
            Assignee: Josh Mahonin
             Fix For: 4.7.0

The current implementation had the 'DATE' datatype bound to a Spark SQL 'TimestampType', which
causes a casting error trying to convert from java.sql.Date to java.sql.Timestamp when using
the DataFrame API with Phoenix DATE columns.

This patch modifies the schema handling to treat DATE columns as the Spark 'DateType' instead.
Note that Spark *drops* the hour, minute and second values from these when interfacing using
DataFrames. This follows the java.sql.Date spec, but might not useful to folks who rely on
the hour/minute/second fields working using the DataFrame API and DATE columns. A future improvement
would be to force these to be TimestampTypes instead to preserve information, but it's less
intuitive and probably shouldn't be the default behaviour.

This message was sent by Atlassian JIRA

View raw message