hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Wohlstadter (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-19723) Arrow serde: "Unsupported data type: Timestamp(NANOSECOND, null)"
Date Sat, 09 Jun 2018 04:25:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506809#comment-16506809
] 

Eric Wohlstadter edited comment on HIVE-19723 at 6/9/18 4:24 AM:
-----------------------------------------------------------------

[~teddy.choi]

Serializer needs to create a {{TimeStampMicroTZVector}} instead of {{TimeStampMicroVector}}. 

See: {{org.apache.spark.sql.vectorized.ArrowColumnVector.ArrowColumnVector(ValueVector vector)}}

Can you create a new JIRA for that?


was (Author: ewohlstadter):
[~teddy.choi]

Serializer needs to create a {{TimeStampMicroTZVector}} instead of {{TimeStampMicroVector}}. 

See: {{org.apache.spark.sql.vectorized.ArrowColumnVector.ArrowColumnVector(ValueVector vector)}}

> Arrow serde: "Unsupported data type: Timestamp(NANOSECOND, null)"
> -----------------------------------------------------------------
>
>                 Key: HIVE-19723
>                 URL: https://issues.apache.org/jira/browse/HIVE-19723
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Teddy Choi
>            Assignee: Teddy Choi
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.1.0
>
>         Attachments: HIVE-19723.1.patch, HIVE-19723.3.patch, HIVE-19723.4.patch, HIVE-19732.2.patch
>
>
> Spark's Arrow support only provides Timestamp at MICROSECOND granularity. Spark 2.3.0
won't accept NANOSECOND. Switch it back to MICROSECOND.
> The unit test org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow will just need to change
the assertion to test microsecond. And we'll need to add this to documentation on supported
datatypes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message