impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Jacobs (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5539) Reading timestamps from Kudu are wrong with -use_local_tz_for_unix_ts
Date Wed, 19 Jul 2017 22:27:00 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthew Jacobs resolved IMPALA-5539.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0

commit 7a1ff1e5e9302ad9c3bf287d414436cc0d3046cf
Author: Matthew Jacobs <mj@cloudera.com>
Date:   Sat Jun 24 14:42:59 2017 -0700

    IMPALA-5539: Fix Kudu timestamp with -use_local_tz_for_unix_ts
    
    The -use_local_tz_for_unix_timestamp_conversion flag exists
    to specify if TIMESTAMPs should be interpreted as localtime
    or UTC when converting to/from Unix time via builtins:
      from_unixtime(bigint unixtime)
      unix_timestamp(string datetime[, ...])
      unix_timestamp(timestamp datetime)
    
    However, the KuduScanner was calling into code that, when
    the gflag above was set, interpreted Unix times as local
    time.  Unfortunately the write path (KuduTableSink) and some
    FE TIMESTAMP code (see KuduUtil.java) did not have this
    behavior, i.e. we were handling the gflag inconsistently.
    
    Tests:
    * Adds a custom cluster test to run Kudu test cases with
      -use_local_tz_for_unix_timestamp_conversion.
    * Adds tests for the new builtin
      unix_micros_to_utc_timestamp() which run in a custom
      cluster test (added test_local_tz_conversion.py) as well
      as in the regular tests (added to test_exprs.py).
    
    Change-Id: I423a810427353be76aa64442044133a9a22cdc9b
    Reviewed-on: http://gerrit.cloudera.org:8080/7311
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins


> Reading timestamps from Kudu are wrong with -use_local_tz_for_unix_ts
> ---------------------------------------------------------------------
>
>                 Key: IMPALA-5539
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5539
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.9.0
>            Reporter: yyzzjj
>            Assignee: Matthew Jacobs
>            Priority: Critical
>              Labels: Kudu, timestamp
>             Fix For: Impala 2.10.0
>
>
> our impalad  start with  flag  use_local_tz_for_unix_timestamp_conversions = true
> so  be  timeconversion  use local time zone
>  Frontend   KuduUtil call  utc_to_unix_micros  function   got time (not handle local
time zone branch , time is not equal  be time conversion TimestampValue::ToUnixTime)
> {code:c++}
> // Some comments here
> inline bool TimestampValue::UtcToUnixTime(time_t* unix_time) const {
>   DCHECK(unix_time != nullptr);
>   if (UNLIKELY(!HasDateAndTime())) return false;
>   const boost::posix_time::ptime temp(date_, time_);
>   tm temp_tm = boost::posix_time::to_tm(temp);
>   // TODO: Conversion using libc is very expensive (IMPALA-5357); find an alternative.
>     *unix_time = timegm(&temp_tm);
>   return true;
> }
> {code}
> beijing/china  is UTC + 8, return wrong result



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message