avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fengyun RAO <raofeng...@gmail.com>
Subject Is there a way to write spark RDD to Avro files
Date Wed, 30 Jul 2014 09:14:32 GMT
We used mapreduce for ETL and storing results in Avro files, which are
loaded to hive/impala for query.

Now we are trying to migrate to spark, but didn't find a way to write
resulting RDD to Avro files.

I wonder if there is a way to make it, or if not, why spark doesn't support
Avro as well as mapreduce? Are there any plans?

Or what's the recommended way to output spark results with schema? I don't
think plain text is a good choice.

Mime
View raw message