drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vlad Rozov <vro...@apache.org>
Subject Re: Parquet MR version
Date Tue, 09 Jan 2018 23:26:29 GMT
Support for byte buffers aka "PARQUET-77: ByteBuffer use in read and 
write paths" is included both to 1.8.2 and 1.9.0 parquet-mr. Based on 
the following discussion 
https://lists.apache.org/thread.html/e1a3d1e49d3b775cb5e1417c882dad7bd2f038e00237ff2a9057c608@1454714357@%3Cdev.drill.apache.org%3E

Drill should move to the released version of parquet-mr. I'll file a 
JIRA and summarize my findings.

Thank you,

Vlad

On 1/5/18 02:23, salim achouche wrote:
> I don’t think it will be a lot of work to do so as the main changes (from the Parquet
branch) is related to passing direct buffers vs using Java heap.
>
> FYI - I would also think that we should contribute to the Parquet code base so that an
abstraction is used on top of byte arrays (e.g., ByteBuffer) so that Drill will not need to
use a private Parquet branch. I guess the key is to figure out why such an approach was not
followed in the first place.
>
> Regards,
> Salim
>
>> On Jan 5, 2018, at 2:37 AM, Vlad Rozov <vrozov@apache.org> wrote:
>>
>> Hi everyone,
>>
>> With parquet-mr 1.9.0 released more than 1 year ago and parquet-mr 1.8.2 almost a
year ago, should the drill dependency on parquet-mr be updated from the current custom 1.8.1-drill-r0?
As far as I can tell both 1.8.2 and 1.9.0 have fixes from the 1.8.1-drill-r0 patch. Will the
community prefer 1.9.0 over 1.8.2 even if 1.9.0 may introduce some incompatibilities?
>>
>> Thank you,
>>
>> Vlad


Mime
View raw message