spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Nastetsky <>
Subject spark-avro 2.0.1 generates strange schema (spark-avro 1.0.0 is fine)
Date Wed, 14 Oct 2015 20:38:12 GMT
I save my dataframe to avro with spark-avro 1.0.0 and it looks like this
(using avro-tools tojson):


But when I use spark-avro 2.0.1, it looks like this:


At this point I'd be happy to use spark-avro 1.0.0, except that it doesn't
seem to support specifying a compression codec (I want deflate).

View raw message