Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E9ABB200D08 for ; Wed, 23 Aug 2017 02:44:47 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E1553167EF3; Wed, 23 Aug 2017 00:44:47 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0BEA9167EF2 for ; Wed, 23 Aug 2017 02:44:46 +0200 (CEST) Received: (qmail 2482 invoked by uid 500); 23 Aug 2017 00:44:44 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 2470 invoked by uid 99); 23 Aug 2017 00:44:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Aug 2017 00:44:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BE4BBC3131; Wed, 23 Aug 2017 00:44:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.749 X-Spam-Level: *** X-Spam-Status: No, score=3.749 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, KAM_LOTSOFHASH=0.25, KAM_NUMSUBJECT=0.5, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id mh6iTYQDqlYp; Wed, 23 Aug 2017 00:44:42 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id B196A5F4A9; Wed, 23 Aug 2017 00:44:41 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 57FDAE0059; Wed, 23 Aug 2017 00:44:39 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id EC601C40019; Wed, 23 Aug 2017 00:44:36 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5724944169554680098==" MIME-Version: 1.0 Subject: Re: Review Request 61188: HIVE-16614 From: Ashutosh Chauhan To: Ashutosh Chauhan Cc: hive , =?utf-8?q?Jes=C3=BAs_Camacho_Rodr=C3=ADguez?= Date: Wed, 23 Aug 2017 00:44:36 -0000 Message-ID: <20170823004436.1763.91247@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Ashutosh Chauhan X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/61188/ X-Sender: Ashutosh Chauhan References: <20170727173333.56760.17395@reviews-vm2.apache.org> In-Reply-To: <20170727173333.56760.17395@reviews-vm2.apache.org> X-ReviewBoard-Diff-For: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveType.java X-ReviewBoard-Diff-For: common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java X-ReviewBoard-Diff-For: ql/src/test/queries/clientpositive/timezone.q X-ReviewBoard-Diff-For: ql/src/test/results/clientpositive/timezone.q.out X-ReviewBoard-Diff-For: serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TimestampTZTypeInfo.java Reply-To: Ashutosh Chauhan X-ReviewRequest-Repository: hive-git archived-at: Wed, 23 Aug 2017 00:44:48 -0000 --===============5724944169554680098== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61188/#review183557 ----------------------------------------------------------- common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java Lines 55 (patched) This may come from config to allow parsing of different formats. common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java Lines 57 (patched) This may come from config to allow parsing of different formats. common/src/java/org/apache/hadoop/hive/conf/HiveConf.java Lines 1028 (patched) Better name: hive.session.time.zone ? ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java Line 318 (original), 321 (patched) We shall write TZ. ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java Lines 330 (patched) and read it back here, instead from conf. ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java Line 966 (original), 966 (patched) missing Timestamp_Type_Name ? ql/src/test/queries/clientpositive/timezone.q Lines 59 (patched) Can you also add tests for join, gby, ptf on ts w/ tz type. So, that we can test serialization from Map to Reducer? ql/src/test/queries/clientpositive/timezone.q Lines 66 (patched) Can you also add tests for casts tstz -> date date -> tstz tstz -> ts ts -> tstz serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java Line 432 (original), 433 (patched) Worth adding a comment: Across Map/Reduce process boundary tz is normalized and stored in type and is not carried in data for each row. serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java Line 74 (original) Not needed anymore? serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java Line 116 (original) Not needed anymore? serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java Lines 304-307 (patched) Add comments for this. - Ashutosh Chauhan On July 27, 2017, 5:33 p.m., Jesús Camacho Rodríguez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61188/ > ----------------------------------------------------------- > > (Updated July 27, 2017, 5:33 p.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Bugs: HIVE-16614 > https://issues.apache.org/jira/browse/HIVE-16614 > > > Repository: hive-git > > > Description > ------- > > HIVE-16614 > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java ed83871a4a049a3e1a8417fb68bd1a1e66026a4e > common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java PRE-CREATION > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 05f6cc95927c130100df01708a3fd186e1cfd116 > common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 0cef77a9cee263b98495c4d139d978364d9320b0 > jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 6742423ff509c8098ad821540ff778a0db2dd6a7 > ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 8902f6c2db72d9804f34b9006d867d85d54ee916 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 517ce312a7783d32b1a9fe91f22ac2293279c112 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveType.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ExprNodeConverter.java f974cc9195772e09b5d09b4da6adf2919ceb529f > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 7665f56cfccf6c92e0fb7d03e475a4db6822f9e0 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java c6b34d46c44591a7d6e06598426414f13589359f > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 2df7588cba01399ae980059a7b2f447a9973f2e6 > ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 5b7fc25417e0a21833b47f4fb399c1a780642a13 > ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 632b9c62cfeb34e9e1cad17ad5f522fe06b882b0 > ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java 1458211b81514963975269ea0dcd33276d851c28 > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java e96012bbf2babc3b9f200a61862540fbf5ded999 > ql/src/test/queries/clientpositive/timestamptz_2.q a335f529d7bff9e2495ff10fd38d30e2d77bb909 > ql/src/test/queries/clientpositive/timezone.q PRE-CREATION > ql/src/test/results/clientpositive/annotate_stats_select.q.out 67d134ba4a9cd8db12513d7de8bb9d03b52a4f3d > ql/src/test/results/clientpositive/constantfolding.q.out 10e185f00b8a8f848122e604906a72dd54ee9d3d > ql/src/test/results/clientpositive/timestamptz.q.out 626fe92286651560a7fb0d6dca98584d49fb671f > ql/src/test/results/clientpositive/timestamptz_1.q.out 75bbfac3e816e3979a72726b696e88e98d7d1e73 > ql/src/test/results/clientpositive/timestamptz_2.q.out 2666735fbc4660723598c6ef70e5913185118328 > ql/src/test/results/clientpositive/timezone.q.out PRE-CREATION > serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java f333ae9938b5f17925c1e37024e7d3b85037a990 > serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampTZWritable.java 8c3f8f647152c4447ef0ee5255d6b8b4727406d0 > serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySerDeParameters.java ee4bb345cee4d0bf6f7952c5e8549ee51059a0ec > serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestampTZ.java df5c586f56274f722f8779afb4bb6262b61d5f02 > serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 6d1ee1e97bb02a53774d272a22b493fa65136800 > serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampTZObjectInspector.java 7336385a7aea4187d729694437fb32b6f88229a5 > serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestampTZ.java 6d9ca6e93781f9aafca91be203bed7c4c078e76a > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java ca96e33600bedac8363b3a78e36898a8297c410b > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampTZObjectInspector.java 32b9c69909488dc4e32cf4ba84d7b97881c4d926 > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java d4b7a32bcaebb4b8402286b970a3fe388f31474a > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 2425c30f012fd536245f6e333891decd0abf98c6 > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 886c29885241ecf55532e030120e23f9fc19145c > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantTimestampTZObjectInspector.java 5805ce8b5e1512d43463017a4fba5be3fa496820 > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampTZObjectInspector.java 0b622c1ed5a268979ff2e04fc9e1d3ce0e1d78f9 > serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TimestampTZTypeInfo.java PRE-CREATION > serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java b0e0bf28be82eec23ad285e8f8f785d6f503fb60 > serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java 8be689653229b01d95f3610bc2c023ff65916e79 > serde/src/test/org/apache/hadoop/hive/serde2/io/TestTimestampTZWritable.java 73b81b9cb578fd391bde318a957763ec31aecfcf > > > Diff: https://reviews.apache.org/r/61188/diff/1/ > > > Testing > ------- > > > Thanks, > > Jesús Camacho Rodríguez > > --===============5724944169554680098==--