orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dain Sundstrom <d...@iq80.com>
Subject WriterOptions.writerVersion(version)?
Date Wed, 27 Feb 2019 17:43:16 GMT
Hi, we recently updated to Hive 3.0+ and have noticed some issues with older versions of Hive
being able to read data written by newer versions of ORC.  Specifically, older readers only
understand writer version up to 4 and newer versions write 6.  This causes older readers to
fail.  I see that the workaround is to set `WriterOptions.writerVersion(WriterVersion.HIVE_13083)`,
which causes the writer to put a `4` in the postscript, but doesn’t seem to change anything
else in the writer’s behavior.  My question is, did I miss something gin the writer where
behavior changes based on version?  If not, does that work?  I ask because newer versions
have comments like `ORC_135(6) => timestamp stats use utc`, which to me would seem to require
that the behavior changes.



Dain Sundstrom
Co-founder @ Presto Software Foundation, Co-creator of Presto (https://prestosql.io)

View raw message