spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Uang <justin.u...@gmail.com>
Subject UDTs and StringType upgrade issue for Spark 1.4.0
Date Fri, 22 May 2015 15:14:41 GMT
We ran into an issue regarding Strings in UDTs when upgrading to Spark
1.4.0-rc. I understand that it's a non-public APIs, so it's expected, but I
just wanted to bring it up for awareness and so we can maybe change the
release notes to mention them =)

Our UDT was serializing to a StringType, but now strings are represented
internally as UTF8String, so we had to change our UDT to use
UTF8String.apply() and UTF8String.toString() to convert back to String.

Mime
View raw message