hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shyam Sundar Sarkar (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HIVE-192) Cannot create table with timestamp type column
Date Mon, 11 May 2009 21:08:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708221#action_12708221
] 

Shyam Sundar Sarkar edited comment on HIVE-192 at 5/11/09 2:08 PM:
-------------------------------------------------------------------

Dear Ashish and others,

After inspecting many different options for implementation, following implementation seems
have the right direction::

(1)  TIMESTAMP values cannot be earlier than 1970 UTC or later than '2038-01-09 03:14:07'
UTC. This means that 
       a date such as '1968-01-01', while legal as a DATETIME or DATE value, is not valid
as a TIMESTAMP value 
       and is converted to 0. 

(2)  The display width is fixed at 19 characters, and the format is 'YYYY-MM-DD HH:MM:SS'.
 TIMESTAMP values 
       are converted from the current time zone to UTC for storage, and converted back from
UTC to the current 
       time zone for retrieval. By default, the current time zone for each connection is the
server's time.

(3)  The implementation involves conversion of TIMESTAMP string into UTC which is a long integer
number 
       representing number of seconds from 1970 UTC before storing through SERDE2.  Similarly,
a long integer
       type representing number of seconds from 1970 UTC will be converted in TIMESTAMP string
type after
       retrieval through SERDE2.

(4)  In this implementation SERDE2  or thrift software layer will not be touched at all. 
The metadata store will
       maintain the information about TIMESTAMP type only.  Actual storage will keep long
integer values.

Please comment on this observation and suggest any corrections!

Thanks,
Shyam_sarkar@yahoo.com


      was (Author: ssarkar):
    Dear Ashish and others,

After inspecting many different options for implementation, following implementation seems
have the right direction::

(1)  TIMESTAMP values cannot be earlier than 1970 UTC or later than '2038-01-09 03:14:07'
UTC. This means that 
       a date such as '1968-01-01', while legal as a DATETIME or DATE value, is not valid
as a TIMESTAMP value 
       and is converted to 0. 

(2)  The display width is fixed at 19 characters, and the format is 'YYYY-MM-DD HH:MM:SS'.
 TIMESTAMP values 
       are converted from the current time zone to UTC for storage, and converted back from
UTC to the current 
       time zone for retrieval. By default, the current time zone for each connection is the
server's time.

(3)  The implementation involves conversion of TIMESTAMP string into UTC which is a long integer
number 
       representing number of seconds from 1970 UTC before storing through SERDE2.  Similarly,
a long integer
       type representing number of seconds from 1970 UTC will be converted in TIMESTAMP string
type after
       retrieval through SERDE2.

(4)  In this implementation SERDE2  or thrift software layer will not be touched at all. 
The metadata store will
       maintain the information about TIMESTAMP type only.  Actual storage will keep long
integer values.

Please comment on this observation and suggest and corrections!

Thanks,
Shyam_sarkar@yahoo.com

  
> Cannot create table with timestamp type column
> ----------------------------------------------
>
>                 Key: HIVE-192
>                 URL: https://issues.apache.org/jira/browse/HIVE-192
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.2.0
>            Reporter: Johan Oskarsson
>            Assignee: Shyam Sundar Sarkar
>             Fix For: 0.4.0
>
>         Attachments: create_2.q.txt, TIMESTAMP_specification.txt
>
>
> create table something2 (test timestamp);
> ERROR: DDL specifying type timestamp which has not been defined
> java.lang.RuntimeException: specifying type timestamp which has not been defined
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.FieldType(thrift_grammar.java:1879)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.Field(thrift_grammar.java:1545)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.FieldList(thrift_grammar.java:1501)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.Struct(thrift_grammar.java:1171)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.TypeDefinition(thrift_grammar.java:497)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.Definition(thrift_grammar.java:439)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.thrift_grammar.Start(thrift_grammar.java:101)
> 	at org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.initialize(DynamicSerDe.java:97)
> 	at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:180)
> 	at org.apache.hadoop.hive.ql.metadata.Table.initSerDe(Table.java:141)
> 	at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:202)
> 	at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:641)
> 	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:98)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:215)
> 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:174)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:207)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:305)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message