spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: Does feature parity exist between Spark and PySpark
Date Wed, 07 Oct 2015 17:03:05 GMT
>
> At my company we use Avro heavily and it's not been fun when i've tried to
> work with complex avro schemas and python. This may not be relevant to you
> however...otherwise I found Python to be a great fit for Spark :)
>

Have you tried using https://github.com/databricks/spark-avro ? It should
help you avoid needing to work with avro directly by translating to/from
DataFrames.

Mime
View raw message