orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Groffen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ORC-526) orc-tools convert does not respect second fractions
Date Thu, 27 Jun 2019 11:31:00 GMT
Fabian Groffen created ORC-526:

             Summary: orc-tools convert does not respect second fractions
                 Key: ORC-526
                 URL: https://issues.apache.org/jira/browse/ORC-526
             Project: ORC
          Issue Type: Bug
          Components: tools
    Affects Versions: 1.5.5, 1.2.0, 1.6.0
            Reporter: Fabian Groffen

{{% cat timestamp2.csv }}
{{2019-01-01 00:00:00.0000}}
{{2015-01-01 00:00:00.0001}}
{{2015-01-01 00:00:00.0000}}
{{2014-12-31 23:59:59.9999}}
{{1970-01-01 00:00:00.0001}}
{{1970-01-01 00:00:00.0000}}
{{1969-12-31 23:59:59.9999}}
{{1969-12-31 23:59:59.0001}}
{{1969-12-31 23:59:59.0000}}
{{1969-12-31 23:59:58.9999}}
{{% java -jar tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar convert --schema timestamp --timestampformat
"yyyy-MM-dd HH:mm:ss.SSSS" timestamp2.csv -o timestamp2-tools16.orc}}

Reading back the file generated above excludes the second fractions (nanos).


Reason for this is that tools/convert/CsvReader.java (as well as tools/convert/JsonReader.java)
do not consider the nanos.  They just create a Timestamp and never call setNanos() method
on it (in this case with the result of a getNano() call to the ZonedDateTime.  This needs
to be done in 4 places.  Unfortunately I am not allowed to make a contribution at this point,
so I can only describe the problem.  I can confirm with a change/fix like that, the nanos
are stored in the resulting ORC file.

This message was sent by Atlassian JIRA

View raw message