Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
MIME-Version: 1.0
From: Zachary Amsden <zamsden@cloudera.com>
Date: Tue, 31 Oct 2017 14:48:53 -0700
Message-ID: <CAA+206fk6i8Q7r-2kS_ACH0ZAARuhbqfNfJbBfFe8w9N8oJ38w@mail.gmail.com>
Subject: IMPALA-5607
To: "dev@impala" <dev@impala.incubator.apache.org>, jinchul@gmail.com
Cc: Greg Rahn <grahn@cloudera.com>
Content-Type: multipart/alternative; boundary="001a11465d5aa05eaa055cdeb851"
archived-at: Tue, 31 Oct 2017 21:49:02 -0000

--001a11465d5aa05eaa055cdeb851
Content-Type: text/plain; charset="UTF-8"

In discussion about IMPALA-5607, it came up that implementing this as
described is a compatibility breaking change.  There are a couple of
required changes:

1) Return data type for date_part and EXTRACT FROM must be promoted to
BIGINT.
2) MILLISECONDS now will include seconds part in the calculation of
milliseconds.

I think the first is a non-issue; we're promoting the type to be wide
enough to hold nanoseconds precision values (with the seconds component
included).  An alternative could be to return this as a decimal type, but
that seems rather unwieldy for other date expressions so I'd prefer these
values to all be returned as integral types.

The bigger issue is including seconds in the calculation of milliseconds,
microseconds and nanoseconds breaks the existing value returned for
milliseconds, which is just bare milliseconds with no seconds component.
For compatibility with other SQL implementations, I think we'd like to
include seconds with all of these date parts, but that is certainly
debatable.

The question then, is anyone relying on this functionality that can't
easily workaround such a change?  The Impala documentation doesn't specify
this behavior either way, and there isn't a formal specification for how
sub-second granularity time is handled.  Whatever we decide, we should
document this going forward.

 - Zach

--001a11465d5aa05eaa055cdeb851--