crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Whitacre <mkwhita...@gmail.com>
Subject Re: crunch+protobuf+parquet
Date Tue, 10 Jun 2014 15:06:04 GMT
Currently there is no support for writing out a Parquet file using protobuf
only Avro.  Seems like something we could support so feel free to long an
enhancement.[1]  Are you looking for some functionality specific to
protobuf with Parquet?  I haven't done it but I believe there are utilities
to store protobuf as Avro[2] so that is an option unless you have a
specific requirement to need to store in the Parquet Protobuf format.

[1] - https://issues.apache.org/jira/browse/CRUNCH
[2] -
http://avro.apache.org/docs/1.6.1/api/java/org/apache/avro/protobuf/ProtobufData.html


On Tue, Jun 10, 2014 at 3:13 AM, Kristoffer Sjögren <stoffe@gmail.com>
wrote:

> Hi
>
> Are there source and target implementations available for using
> protobuf (instead of avro) with parquet in crunch?
>
> Cheers,
> -Kristoffer
>

Mime
View raw message