Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201710041519.v94FJgaj024485@ip-10-146-233-104.ec2.internal>
Date: Wed, 4 Oct 2017 15:19:42 +0000
From: "Csaba Ringhofer (Code Review)" <gerrit@cloudera.org>
To: Lars Volker <lv@cloudera.com>, impala-cr@cloudera.com,
        reviews@impala.incubator.apache.org
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5664=3A_Unix_time_to_timestamp_conversions_may_crash_Impala=0A?=
In-Reply-To: <gerrit.1504538493000.I77b2f6284d3a597f57e61c17a67c959eff9e38ff@gerrit.cloudera.org>
References: <gerrit.1504538493000.I77b2f6284d3a597f57e61c17a67c959eff9e38ff@gerrit.cloudera.org>
Reply-To: csringhofer@cloudera.com, impala-cr@cloudera.com, lv@cloudera.com,
        marcelk@gmail.com, reviews@impala.incubator.apache.org,
        dhecht@cloudera.com
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="b1Ky5A4whaw="; charset=UTF-8
archived-at: Wed, 04 Oct 2017 15:19:49 -0000

--b1Ky5A4whaw=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello Lars Volker, Dan Hecht, 

I'd like you to reexamine a change=2E Pleas=
e visit

    http://gerrit=2Ecloudera=2Eorg:8080/7954

to look at the new p=
atch set (#11)=2E

Change subject: IMPALA-5664: Unix time to timestamp conv=
ersions may crash Impala
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E

IMPALA-5664: Unix time to timestamp conversions may crash Imp=
ala

TimestampValue::FromSubsecondUnixTime() and UtcFromUnixTimeMicros()
ar=
e incorrect only in case of the last second of 1399, because these
sub-seco=
nd values are rounded first towards 1400-01-01 00:00:00, which
is accepted =
as a valid date, and the sub-second part is subtracted
afterwards, leading =
to a date outside the valid interval=2E The maximum
case, 9999-12-31 59:59:=
59 is a bit different, because as I understand,
with nanosecond precision p=
osix times, the maximum value is actually
10000-01-01=2E 00:00:00 minus 1 n=
anosec=2E

TimestampValue::FromUnixTimeNanos() can create problematic Times=
tampValues
both <1400 and 10000<=3D=2E

These timestamps can cause problems=
, because most code assumes that if
HasDate/HasTime is true, then it really=
 is a valid timestamp=2E

To fix this, the posix times are checked in the c=
onstructor of
TimestampValue, and if it is outside the valid interval,both =
time_
and date_ are set to not_a_date_time=2E

Also added logging to "cast =
to timestamp" functions=2E

Test:
select cast(-17987443200-0=2E1 as timesta=
mp);
This query no longer crashes, but returns NULL, similarly to other
< 1=
400 timestamps=2E

Change-Id: I77b2f6284d3a597f57e61c17a67c959eff9e38ff
---=

M be/src/exec/parquet-column-readers=2Ecc
M be/src/exec/parquet-column-sta=
ts=2Einline=2Eh
M be/src/exprs/cast-functions-ir=2Ecc
M be/src/exprs/timest=
amp-functions-ir=2Ecc
M be/src/runtime/timestamp-test=2Ecc
M be/src/runtime=
/timestamp-value=2Ecc
M be/src/runtime/timestamp-value=2Eh
7 files changed,=
 66 insertions(+), 10 deletions(-)


  git pull ssh://gerrit=2Ecloudera=2Eo=
rg:29418/Impala-ASF refs/changes/54/7954/11
-- 
To view, visit http://gerri=
t=2Ecloudera=2Eorg:8080/7954
To unsubscribe, visit http://gerrit=2Ecloudera=
=2Eorg:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerr=
it-MessageType: newpatchset
Gerrit-Change-Id: I77b2f6284d3a597f57e61c17a67c=
959eff9e38ff
Gerrit-Change-Number: 7954
Gerrit-PatchSet: 11
Gerrit-Owner: C=
saba Ringhofer <csringhofer@cloudera=2Ecom>
Gerrit-Reviewer: Csaba Ringhofe=
r <csringhofer@cloudera=2Ecom>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera=
=2Ecom>
Gerrit-Reviewer: Lars Volker <lv@cloudera=2Ecom>

--b1Ky5A4whaw=--