hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rui Li <rui...@intel.com>
Subject Re: Review Request 50787: Add a timezone-aware timestamp
Date Fri, 23 Sep 2016 08:58:24 GMT


> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > - How about compatbility with the various date functions (year()/month()/day()/etc)?

For most of the functions, TIMESTAMPTZ is implicitly converted to text. Therefore I think
we can get correct results. I added some special handle in HOUR because some hour may be unavailable
due to DST.
So far I've verified the following funcsions work:

to_date
year
quarter
month
day
dayofmonth
hour
minute
second
weekofyear

Is it OK we leave others in follow-on tasks? I'd like to keep the patch small.


> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java,
line 58
> > <https://reviews.apache.org/r/50787/diff/4/?file=1507773#file1507773line58>
> >
> >     No conversions to/from DATE/TIMESTAMP?

Added conversion from date/timestamp to timestamptz. Default timezone is used for the converted
timestamptz.
We can add convertion from numeric types in follow-on task.


> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java,
line 1109
> > <https://reviews.apache.org/r/50787/diff/4/?file=1507829#file1507829line1109>
> >
> >     If the local timezone is different from the timezone in the TimestampTZ, is
it possible that the year/month/day of the DATE might be different from the year/month/day
of the TimestampTZ?

Good catch! It makes more sense to convert from the text representation than the time/nanos.
So I convert the timestamptz to string first, and use that string to create the date. Same
applies when converting to timestamp.


- Rui


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review149983
-----------------------------------------------------------


On Sept. 22, 2016, 4:05 a.m., Rui Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> -----------------------------------------------------------
> 
> (Updated Sept. 22, 2016, 4:05 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
>     https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -----
> 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java PRE-CREATION

>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 7be628e 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
ba41518 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 8b0db4a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82080eb 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java PRE-CREATION

>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
>   ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
>   ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
>   ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
>   ql/src/test/results/clientpositive/create_like.q.out 0111c94 
>   ql/src/test/results/clientpositive/join43.q.out 127d5d0 
>   ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
>   ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
>   serde/if/serde.thrift 6caad36 
>   serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
>   serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
>   serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
04ed8f5 
>   serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 18c3991 
>   serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py fafdc24 
>   serde/src/gen/thrift/gen-rb/serde_constants.rb 0ce9f27 
>   serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 7ffc964 
>   serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
5e119d7 
>   serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/fast/BinarySortableDeserializeRead.java
a7785b2 
>   serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampTZWritable.java PRE-CREATION

>   serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java bbccc7f 
>   serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritableBase.java PRE-CREATION

>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 23dbe6a 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java 56945d1 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestampTZ.java PRE-CREATION

>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 73c72e1 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
5601734 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampTZObjectInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 52f3527

>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 54bfd2d

>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestampTZ.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java f8a110d

>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
24b3d4e 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
1ac72c6 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java
70633f3 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java
509189e 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampTZObjectInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java
e08ad43 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java
2ed0843 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
51b529e 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampTZObjectInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampTZObjectorInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantTimestampTZObjectInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampTZObjectInspector.java
PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/thrift/Type.java 0ad8c02 
>   serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 43c4819

>   serde/src/test/org/apache/hadoop/hive/serde2/io/TestTimestampWritable.java 3c483cc

>   service-rpc/if/TCLIService.thrift a4fa7b0 
>   service-rpc/src/gen/thrift/gen-cpp/TCLIService_constants.cpp 991cb2e 
>   service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.h b249544 
>   service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.cpp 2f460e8 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TCLIServiceConstants.java
930bed7 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TProtocolVersion.java
bce2a0c 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TTypeId.java
a3735eb 
>   service-rpc/src/gen/thrift/gen-php/Types.php 786c773 
>   service-rpc/src/gen/thrift/gen-py/TCLIService/constants.py c8d4f8f 
>   service-rpc/src/gen/thrift/gen-py/TCLIService/ttypes.py fdf6b1f 
>   service-rpc/src/gen/thrift/gen-rb/t_c_l_i_service_constants.rb 25adbb4 
>   service-rpc/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb 4b1854c 
>   service/src/java/org/apache/hive/service/cli/ColumnValue.java 76e8c03 
>   service/src/java/org/apache/hive/service/cli/TypeDescriptor.java d634bef 
>   storage-api/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java PRE-CREATION

>   storage-api/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java 4a745e4 
>   storage-api/src/java/org/apache/hadoop/hive/ql/util/TimestampUtils.java 41db9ca 
> 
> Diff: https://reviews.apache.org/r/50787/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rui Li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message