crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <>
Subject [jira] [Assigned] (CRUNCH-370) Update Parquet dependency in Crunch pom
Date Wed, 26 Mar 2014 03:09:15 GMT


Micah Whitacre reassigned CRUNCH-370:

    Assignee: Micah Whitacre

> Update Parquet dependency in Crunch pom
> ---------------------------------------
>                 Key: CRUNCH-370
>                 URL:
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>    Affects Versions: 0.9.0
>            Reporter: Anandsagar Kothapalli
>            Assignee: Micah Whitacre
> Currently crunch is supporting avro to parquet conversion using AvroParquetFileTarget,
AvroParquetFileSource classes. When I used these classes to convert avro to parquet files,
I got the following exception in some cases: "org.apache.crunch.CrunchRuntimeException:
empty fields are illegal, the field should be ommited completely instead"
> After further debugging I found out that this issue is related to AvroWriteSupport class
in parquet, which was fixed as part of milestone 1.2.3
Latest parquet version is 1.3.2.
> But crunch is still using parquet 1.2.0

> As part of this improvement, parquet dependency version in crunch will be updated if
not to latest then at least to 1.2.3 

This message was sent by Atlassian JIRA

View raw message