crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristoffer Sjögren <sto...@gmail.com>
Subject Re: crunch+protobuf+parquet
Date Wed, 11 Jun 2014 11:37:21 GMT
Ok, I will have a look at the utility first, to see if it fit my
needs, or otherwise log an enhancement.

Thanks for the info!

On Tue, Jun 10, 2014 at 5:06 PM, Micah Whitacre <mkwhitacre@gmail.com> wrote:
> Currently there is no support for writing out a Parquet file using protobuf
> only Avro.  Seems like something we could support so feel free to long an
> enhancement.[1]  Are you looking for some functionality specific to protobuf
> with Parquet?  I haven't done it but I believe there are utilities to store
> protobuf as Avro[2] so that is an option unless you have a specific
> requirement to need to store in the Parquet Protobuf format.
>
> [1] - https://issues.apache.org/jira/browse/CRUNCH
> [2] -
> http://avro.apache.org/docs/1.6.1/api/java/org/apache/avro/protobuf/ProtobufData.html
>
>
> On Tue, Jun 10, 2014 at 3:13 AM, Kristoffer Sjögren <stoffe@gmail.com>
> wrote:
>>
>> Hi
>>
>> Are there source and target implementations available for using
>> protobuf (instead of avro) with parquet in crunch?
>>
>> Cheers,
>> -Kristoffer
>
>

Mime
View raw message