From user-return-1285-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Wed Jun 30 00:37:21 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id B9D8918064E for ; Wed, 30 Jun 2021 02:37:21 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id E878440615 for ; Wed, 30 Jun 2021 00:37:20 +0000 (UTC) Received: (qmail 12948 invoked by uid 500); 30 Jun 2021 00:37:19 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 12938 invoked by uid 99); 30 Jun 2021 00:37:19 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jun 2021 00:37:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id B64D01FF48B for ; Wed, 30 Jun 2021 00:37:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id fUCeLD5-nHa3 for ; Wed, 30 Jun 2021 00:37:17 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::535; helo=mail-ed1-x535.google.com; envelope-from=emkornfield@gmail.com; receiver= Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 466AC7FE7E for ; Wed, 30 Jun 2021 00:37:17 +0000 (UTC) Received: by mail-ed1-x535.google.com with SMTP id t3so660000edc.7 for ; Tue, 29 Jun 2021 17:37:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to; bh=IuJ3SkTytefSexR63EbpnYzFuCQ496xDK2DuMpCGgjc=; b=mJ7gr0c5jRzBN0JlNJZLDNe7X19v3fGezJ3py9t76QnnVYOIC9iAdw973NPNrrWw/d P/pg3zwk+L+gCSYunWl5qxNHf0wDi8Wz8ZhL0bjlJlrqmbaYGv9Nk9Naxx3oA8gEGGpI zjlX0jMmCeXkyWibptA5V9yOCx7QDT3SRDZu93Gfgz/Lf97adliCv9AAr1E/kJlOMYcc AO1X9q7u/6y0WZ1Gp1CI1Ojx2NRJKbxzoC9BRPSSIksiuQMRS+p3ARiXrr51xZ0aQgQo TLtGoQkYhvCj983XFM4+LqZEesytj0aaAMV7RmmNvlv34pxUQeBV6Pnt/ylvPciMzWGo 6gpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to; bh=IuJ3SkTytefSexR63EbpnYzFuCQ496xDK2DuMpCGgjc=; b=d0GqJTgMCAaQ0yj5PAxc768UvXaA37083skmQegiGWv3EQeXYq3Z6WQwcFPzUUF9Sz gn8WKvekzeKLWdMg1u9mTm6ul/Vpxqy3Hu2HlYBai+cR9yiFHubBpVlALsEd//cj/IuK jhAnqWKSrxxXfOWF2UGJvVlMZLO9MLQVmdZMCBTkatxOQscYRkc9JIUqx2N+IOUsG2aM Q5tvLtZG2gBzG+m4ju3ZapnUE03O32rK0bj/YwcjIA0NoszwUuK6iAHA3IcEoc10lAor Grw0zLS4a8/ksi9uwl5B3jd6Q/qLl+g6pAJ/wxFXjJiWlHpNKqY9u79FR8tKBOG89EP/ BJCQ== X-Gm-Message-State: AOAM533l3cp/I62VM3NOEtX1LZX+Q3JIHmW/aClH4JxLBnbLzIMyxa3Z cqneN8n/SS28iHzWgxCR5DO5o+NQHhIFyk1KvKgXKSpi X-Google-Smtp-Source: ABdhPJyNgDXNsDDi9X2Ia3bbY0OhTVOctU1+wTwrKFDK13XTgpw2ZfySuS6PvfWxZfDwLW+43+AN+Zml4/u6LkNNH/Q= X-Received: by 2002:aa7:c7c2:: with SMTP id o2mr22878811eds.166.1625013430759; Tue, 29 Jun 2021 17:37:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: Reply-To: emkornfield@gmail.com From: Micah Kornfield Date: Tue, 29 Jun 2021 17:36:58 -0700 Message-ID: Subject: Re: Release date for Arrow 5.0 and LZ4_RAW option To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="0000000000004bb60205c5f0ece9" --0000000000004bb60205c5f0ece9 Content-Type: text/plain; charset="UTF-8" Unless someone has implemented the corresponding changes in Parquet-MR it will not be compatible with hadoop (I haven't been paying close attention but I don't recall seeing a PR adding support for parquet-mr). On Tue, Jun 29, 2021 at 5:01 PM Weston Pace wrote: > Right now I believe July is the target. More info here: > > https://lists.apache.org/thread.html/r69825df037e16a0c10add07cdd6c8b3bb0010bbc18ba56bfdc1c5df8%40%3Cdev.arrow.apache.org%3E > > On Tue, Jun 29, 2021 at 1:51 PM Yuan, Hangjian wrote: > > > > Hi Apache Arrow community, > > > > > > > > Thank you for the effort of making such a great open-source framework. :) > > > > > > > > I found Parquet C++ lib in Apache Arrow 5.0 will provide LZ4_RAW option > for compressing parquet files, which should be compatible with Hadoop. Can > I get a rough release date for version 5 or LZ4_RAW option? > > > > > > > > Thanks, > > > > Hangjian > --0000000000004bb60205c5f0ece9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Unless someone has implemented the corresponding changes i= n Parquet-MR it will not be compatible with hadoop (I haven't been payi= ng close attention but I don't recall seeing=C2=A0a PR adding support f= or parquet-mr).

On Tue, Jun 29, 2021 at 5:01 PM Weston Pace <weston.pace@gmail.com> wrote:
Right now I believe July= is the target.=C2=A0 More info here:
https://lists.apache.org/thread.html/r69825df037e16a= 0c10add07cdd6c8b3bb0010bbc18ba56bfdc1c5df8%40%3Cdev.arrow.apache.org%3E=

On Tue, Jun 29, 2021 at 1:51 PM Yuan, Hangjian <hjy@amazon.com> wrote:
>
> Hi Apache Arrow community,
>
>
>
> Thank you for the effort of making such a great open-source framework.= :)
>
>
>
> I found Parquet C++ lib in Apache Arrow 5.0 will provide LZ4_RAW optio= n for compressing parquet files, which should be compatible with Hadoop. Ca= n I get a rough release date for version 5 or LZ4_RAW option?
>
>
>
> Thanks,
>
> Hangjian
--0000000000004bb60205c5f0ece9--