arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Pitrou <anto...@python.org>
Subject Re: Release date for Arrow 5.0 and LZ4_RAW option
Date Wed, 30 Jun 2021 08:38:49 GMT
On Tue, 29 Jun 2021 17:36:58 -0700
Micah Kornfield <emkornfield@gmail.com> wrote:
> Unless someone has implemented the corresponding changes in Parquet-MR it
> will not be compatible with hadoop (I haven't been paying close attention
> but I don't recall seeing a PR adding support for parquet-mr).

Indeed, a JIRA is open about that:
https://issues.apache.org/jira/browse/PARQUET-2032

I would encourage anyone interested to try and contribute the LZ4_RAW
support in parquet-mr.  I don't know how involved that is (ideally it
should be relatively easy, but that depends on the state of Java
compression libraries, which seems to be a thorny topic).

Regards

Antoine.



Mime
View raw message