crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anandsagar Kothapalli (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-370) Update Parquet dependency in Crunch pom
Date Tue, 25 Mar 2014 14:37:16 GMT


Anandsagar Kothapalli updated CRUNCH-370:

    Component/s:     (was: MapReduce Patterns)

> Update Parquet dependency in Crunch pom
> ---------------------------------------
>                 Key: CRUNCH-370
>                 URL:
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>    Affects Versions: 0.9.0
>            Reporter: Anandsagar Kothapalli
> Currently crunch is supporting avro to parquet conversion using AvroParquetFileTarget,
AvroParquetFileSource classes. When I used these classes to convert avro to parquet files,
I got the following exception in some cases: "org.apache.crunch.CrunchRuntimeException:
empty fields are illegal, the field should be ommited completely instead"
> After further debugging I found out that this issue is related to AvroWriteSupport class
in parquet, which was fixed as part of milestone 1.2.3
Latest parquet version is 1.3.2.
> But crunch is still using parquet 1.2.0

> As part of this improvement, parquet dependency version in crunch will be updated if
not to latest then at least to 1.2.3 

This message was sent by Atlassian JIRA

View raw message