hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udit Mehta <ume...@groupon.com>
Subject Best way to deal with incompatible column type changes
Date Thu, 15 Oct 2015 20:50:57 GMT
Hi,

I have a Hive external table with a lot of partitions where the underlying
data is in JSON. I use this popular serde
<https://github.com/rcongiu/Hive-JSON-Serde> to read and write in JSON
format.

So I have a data stream where sometimes there are changes to the JSON
structure. For eg, a key might change its type from string to a struct or
an array. Replacing/Changing the column via the ALTER statement does not
really help since it results in a ClassCastException(Based on this ticket
<https://issues.apache.org/jira/browse/HIVE-4409>).

My question is what would be the best way to deal with such schema changes
without dropping/creating the table again. Basically i dont want to lose my
partitions.

I am currently using Hive version 0.13.1 but planning to move to version
1.2.1 soon.

Any help/advise would be appreciated.

Thanks,
Udit

Mime
View raw message