From dev-return-14953-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Thu Sep 26 14:12:19 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A6E7E1804BB for ; Thu, 26 Sep 2019 16:12:18 +0200 (CEST) Received: (qmail 41268 invoked by uid 500); 26 Sep 2019 14:12:17 -0000 Mailing-List: contact dev-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@arrow.apache.org Delivered-To: mailing list dev@arrow.apache.org Received: (qmail 41255 invoked by uid 99); 26 Sep 2019 14:12:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Sep 2019 14:12:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8ED20C007C for ; Thu, 26 Sep 2019 14:12:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.052 X-Spam-Level: ** X-Spam-Status: No, score=2.052 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_EXCESS_BASE64=0.001, HTML_MESSAGE=2, KAM_LOTSOFHASH=0.25, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 6F1SSsJrIoHd for ; Thu, 26 Sep 2019 14:12:10 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::32f; helo=mail-ot1-x32f.google.com; envelope-from=szucs.krisztian@gmail.com; receiver= Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id CE7B87F526 for ; Thu, 26 Sep 2019 14:12:08 +0000 (UTC) Received: by mail-ot1-x32f.google.com with SMTP id y39so2109888ota.7 for ; Thu, 26 Sep 2019 07:12:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JOpWM7tjc93osViJTIOFJVTT3nK4WWjJJ6GjlglS8Ac=; b=B7dOB5cGMzLHmX8jUYTLQ7tZDH7rlThEplufw263jv75mco+bz3eXXvFo1ShW7Ajgm W0J/XKJSQxOwRwGB2EZWl9Hn5KYK3WP5T/S+sXjxZXsgzjgMvLigdWnZYLI6bZJlNLsp N1DfKOpX4l7q5OXYHHehsGSUf+9T5ofgQHrRJnfHzFEwpO+aJRZRdck6NswqtBhRjBFZ 2cXD5QLac4jhkQTh/enx2+Aw+o9dYwbf2im7sGzv3x5oNGEJ1CZlSVWgPD3TRM0VPQ73 +SNLW37BwBk+LF7sjukeW80gFoGtwqG1spQlOq10r/zR1tSnicrVVGjjokgoIt293NE/ 0dKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JOpWM7tjc93osViJTIOFJVTT3nK4WWjJJ6GjlglS8Ac=; b=eAWasiDp+PRwJrapkks+n64eLWiNAub1e0NUUwCG4/8r8JmnXftBGSIChEfCW3Adw3 JQCIv8YlJzLAVe3VhFk1WW5kbXjTdYW7pkX4eq0556q4T2em5WVI0Mi+X73nAs6MlsLP T3X0dteVlIcwOtkE1HucontH90UIL5RWPS0gfJG8P0WcnfQg03PNL0SSMkBizZcTeUMU ZmOfG512xwr//6M3Rkar8Ki/tLi+Ij+vUFsxLFGnfOX90IBkpHs16VY2sbAP7Id+ayEe y08YJElNXF13BO9BSmuqu7neHgPSYREzPn9qZ0PiRXxzrux5+b+9OjafNTbaKHhdpjCs bLjA== X-Gm-Message-State: APjAAAXS5nnSgWJ/Onf/gj+Vue5YT8WE1T/GBjhfEaGv0guXhDQ8Z/NG yf9u4bipw5eiTd288ci3ckXWX6oRmktSpe56lI8= X-Google-Smtp-Source: APXvYqxTIloeHBBjjEM3sQ4DpRzgd9Wiyfl5+8F3s3M4lo7pxSDAM/4PJax/NJWrl2TEkKk8wZPyHVIT8Fcq/6KFwSs= X-Received: by 2002:a9d:4919:: with SMTP id e25mr2889981otf.267.1569507121113; Thu, 26 Sep 2019 07:12:01 -0700 (PDT) MIME-Version: 1.0 References: <6a99bfb3-0326-7545-714e-655ac2d229f6@python.org> <20190815202800.251162d8@fsol> <8a32bde5-7ff5-4c3e-b8a6-b82356e0266c.niki.lj@aliyun.com> <20dded22-cfbe-4503-8076-0eeca8c48603.niki.lj@aliyun.com> In-Reply-To: From: =?UTF-8?B?S3Jpc3p0acOhbiBTesWxY3M=?= Date: Thu, 26 Sep 2019 16:11:48 +0200 Message-ID: Subject: Re: Timeline for 0.15.0 release To: Micah Kornfield Cc: Ji Liu , Neal Richardson , Wes McKinney , dev Content-Type: multipart/alternative; boundary="0000000000006d325e0593755a1f" --0000000000006d325e0593755a1f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable There are still missing linux artifacts [1]: - for amd64 debug symbol packages - for arm64 optional CUDA, plasma and Gandiva modules I think we can safely ignore them for the release, crossbow will report them as missing but the artifact downloading step finish. Let me know Micah if you have any issues. [1]: https://github.com/apache/arrow/pull/5506#issuecomment-535495351 On Thu, Sep 26, 2019 at 3:38 PM Micah Kornfield wrote: > Yes, I merged it and it will be included. I needed to start over due to = a > cross-bow issue... > > On Thu, Sep 26, 2019 at 7:18 AM Ji Liu wrote: > >> Hi Micah, >> Hmm, unfortunately, I just found a bug in JDBC adapter and open a >> PR, could this change catch up with 0.15? >> See https://github.com/apache/arrow/pull/5511 >> >> >> Thanks, >> Ji Liu >> >> >> ------------------------------------------------------------------ >> From:Micah Kornfield >> Send Time:2019=E5=B9=B49=E6=9C=8826=E6=97=A5(=E6=98=9F=E6=9C=9F=E5=9B=9B= ) 14:23 >> To:Neal Richardson >> Cc:"Kriszti=C3=A1n Sz=C5=B1cs" ; Wes McKinney= < >> wesmckinn@gmail.com>; dev >> Subject:Re: Timeline for 0.15.0 release >> >> Just an I've started the RC generation process off, the last commit from >> master is [1] >> >> I am currently waiting the crossbow builds (build-690 on >> ursa-labs/crossbow). I think this will take a little while so I will pi= ck >> it up tomorrow (Thursday). >> >> Thanks, >> Micah >> >> [1] >> >> https://github.com/apache/arrow/commit/07ab5083d5a2925ced6f8168b60b8fa33= 6f4eccc >> >> On Wed, Sep 25, 2019 at 2:07 PM Neal Richardson < >> neal.p.richardson@gmail.com> >> wrote: >> >> > IMO it's too risky to add something that adds a dependency >> > (aws-sdk-cpp) on the day of cutting a release. >> > >> > Neal >> > >> > On Wed, Sep 25, 2019 at 12:54 PM Kriszti=C3=A1n Sz=C5=B1cs >> > wrote: >> > > >> > > We don't have a comprehensive documentation yet, so let's postpone i= t. >> > > >> > > >> > > On Wed, Sep 25, 2019 at 9:48 PM Kriszti=C3=A1n Sz=C5=B1cs < >> > szucs.krisztian@gmail.com> wrote: >> > >> >> > >> The S3 python bindings would be a nice addition to the release. >> > >> I don't think we should block on this but the PR is ready. Opinions= ? >> > >> https://github.com/apache/arrow/pull/5423 >> > >> >> > >> >> > >> >> > >> >> > >> On Wed, Sep 25, 2019 at 5:28 PM Micah Kornfield < >> emkornfield@gmail.com> >> > wrote: >> > >>> >> > >>> OK, I'll start the process today. I'll send up e-mail updates as = I >> > make progress. >> > >>> >> > >>> On Wed, Sep 25, 2019 at 8:22 AM Wes McKinney >> > wrote: >> > >>>> >> > >>>> Yes, all systems go as far as I'm concerned. >> > >>>> >> > >>>> On Wed, Sep 25, 2019 at 9:56 AM Neal Richardson >> > >>>> wrote: >> > >>>> > >> >> > >>>> > Andy's DataFusion issue and Wes's Parquet one have both been me= rged, >> >> > >>>> > and it looks like the LICENSE issue is being resolved as I type= . So >> > >>>> > are we good to go now? >> > >>>> > >> > >>>> > Neal >> > >>>> > >> > >>>> > >> > >>>> > On Tue, Sep 24, 2019 at 10:30 PM Andy Grove < >> andygrove73@gmail.com> >> > wrote: >> > >>>> > > >> > >>>> > > I found a last minute issue with DataFusion (Rust) and would >> > appreciate it >> > >>>> > > if we could merge ARROW-6086 (PR is >> > >>>> > > https://github.com/apache/arrow/pull/5494 >> ) before cutting the RC. >> > >>>> > > >> > >>>> > > Thanks, >> > >>>> > > >> > >>>> > > Andy. >> > >>>> > > >> > >>>> > > >> > >>>> > > On Tue, Sep 24, 2019 at 6:19 PM Micah Kornfield < >> > emkornfield@gmail.com> >> > >>>> > > wrote: >> > >>>> > > >> > >>>> > > > OK, I'm going to postpone cutting a release until tomorrow >> > (hoping we can >> > >>>> > > > issues resolved by then).. I'll also try to review the >> > third-party >> > >>>> > > > additions since 14.x. >> > >>>> > > > >> > >>>> > > > On Tue, Sep 24, 2019 at 4:20 PM Wes McKinney < >> > wesmckinn@gmail.com> wrote: >> > >>>> > > > >> > >>>> > > > > I found a licensing issue >> > >>>> > > > > >> > >>>> > > > > https://issues.apache.org/jira/browse/ARROW-6679 >> > >>>> > > > > >> > >>>> > > > > It might be worth examining third party code added to the >> > project >> > >>>> > > > > since 0.14.x to make sure there are no other such issues. >> > >>>> > > > > >> > >>>> > > > > On Tue, Sep 24, 2019 at 6:10 PM Wes McKinney < >> > wesmckinn@gmail.com> >> > >>>> > > > wrote: >> > >>>> > > > > > >> >> > >>>> > > > > > I have diagnosed the problem (Thrift "string" data must= be >> > UTF-8, >> >> > >>>> > > > > > cannot be arbitrary binary) and am working on a patch r= ight >> > now >> > >>>> > > > > > >> > >>>> > > > > > On Tue, Sep 24, 2019 at 6:02 PM Wes McKinney < >> > wesmckinn@gmail.com> >> > >>>> > > > > wrote: >> > >>>> > > > > > > >> > >>>> > > > > > > I just opened >> > >>>> > > > > > > >> > >>>> > > > > > > https://issues.apache.org/jira/browse/ARROW-6678 >> > >>>> > > > > > > >> > >>>> > > > > > > Please don't cut an RC until I have an opportunity to >> > diagnose this, >> > >>>> > > > > > > will report back. >> > >>>> > > > > > > >> > >>>> > > > > > > >> > >>>> > > > > > > On Tue, Sep 24, 2019 at 5:51 PM Wes McKinney < >> > wesmckinn@gmail.com> >> > >>>> > > > > wrote: >> > >>>> > > > > > > > >> > >>>> > > > > > > > I'm investigating a possible Parquet-related >> > compatibility bug >> > >>>> > > > that I >> > >>>> > > > > > > > encountered through some routine testing / >> > benchmarking. I'll >> > >>>> > > > report >> > >>>> > > > > > > > back once I figure out what is going on (if anythin= g) >> > >>>> > > > > > > > >> > >>>> > > > > > > > On Sun, Sep 22, 2019 at 11:51 PM Micah Kornfield < >> > >>>> > > > > emkornfield@gmail.com> wrote: >> > >>>> > > > > > > > >> >> > >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trus= t >> > (i.e. you can >> > >>>> > > > > get it >> > >>>> > > > > > > > >> signed by another PMC member), but is not 100% >> > essential. >> > >>>> > > > > > > > > >> > >>>> > > > > > > > > That won't be an option for me this week (it seem= s >> > like I would >> > >>>> > > > > need to meet one face-to-face). I'll try to get the GPG >> > checked in and >> > >>>> > > > the >> > >>>> > > > > rest of the pre-requisites done tomorrow (Monday) to >> > hopefully start the >> > >>>> > > > > release on Tuesday (hopefully we can solve the last >> > blocker/integration >> > >>>> > > > > tests by then). >> > >>>> > > > > > > > > >> > >>>> > > > > > > > > On Sat, Sep 21, 2019 at 7:12 PM Wes McKinney < >> > >>>> > > > wesmckinn@gmail.com> >> > >>>> > > > > wrote: >> > >>>> > > > > > > > >> >> > >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trus= t >> > (i.e. you can >> > >>>> > > > > get it >> > >>>> > > > > > > > >> signed by another PMC member), but is not 100% >> > essential. >> > >>>> > > > > > > > >> >> > >>>> > > > > > > > >> Speaking of the release, there are at least 2 co= de >> > changes I >> > >>>> > > > still >> > >>>> > > > > > > > >> want to get in >> > >>>> > > > > > > > >> >> > >>>> > > > > > > > >> ARROW-5717 >> > >>>> > > > > > > > >> ARROW-6353 >> > >>>> > > > > > > > >> >> >> > >>>> > > > > > > > >> I just pushed updates to ARROW-5717, will merge = once >> > the build >> > >>>> > > > is >> > >>>> > > > > green. >> > >>>> > > > > > > > >> >> >> > >>>> > > > > > > > >> There are a couple of Rust patches still marked = for >> > 0.15. The >> > >>>> > > > rest >> > >>>> > > > > > > > >> seems to be documentation and a couple of >> > integration test >> > >>>> > > > > failures we >> > >>>> > > > > > > > >> should see about fixing in time. >> > >>>> > > > > > > > >> >> > >>>> > > > > > > > >> On Fri, Sep 20, 2019 at 11:26 PM Micah Kornfield= < >> > >>>> > > > > emkornfield@gmail.com> wrote: >> > >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > Thanks Kriszti=C3=A1n and Wes, >> >> > >>>> > > > > > > > >> > I've gone ahead and started registering myself= on >> > all the >> > >>>> > > > > packaging sites. >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > Is there any review process when adding my GPG= key >> > to the SVN >> > >>>> > > > > file? [1] >> > >>>> > > > > > > > >> > doesn't seem to mention explicitly. >> > >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > Thanks, >> > >>>> > > > > > > > >> > Micah >> > >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > [1] >> > https://www.apache.org/dev/version-control.html#https-svn >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > On Fri, Sep 20, 2019 at 5:01 PM Kriszti=C3=A1n= Sz=C5=B1cs < >> > >>>> > > > > szucs.krisztian@gmail.com> >> > >>>> > > > > > > > >> > wrote: >> > >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney= < >> > >>>> > > > > wesmckinn@gmail.com> wrote: >> > >>>> > > > > > > > >> > > >> > >>>> > > > > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah >> > Kornfield < >> > >>>> > > > > emkornfield@gmail.com> >> > >>>> > > > > > > > >> > >> wrote: >> > >>>> > > > > > > > >> > >> >> >> > >>>> > > > > > > > >> > >> >> The process should be well documented at >> > this point but >> > >>>> > > > > there are a >> > >>>> > > > > > > > >> > >> >> number of steps. >> > >>>> > > > > > > > >> > >> > >> > >>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for t= he >> > release? >> > >>>> > > > Are >> > >>>> > > > > there >> > >>>> > > > > > > > >> > >> instructions for the adding the code signin= g >> > Key to SVN? >> > >>>> > > > > > > > >> > >> > >> > >>>> > > > > > > > >> > >> > I will make a go of it. i will try to >> > mitigate any >> > >>>> > > > > internet issues by >> > >>>> > > > > > > > >> > >> doing the process for a cloud instance (I >> > assume that isn't >> > >>>> > > > > a problem?). >> > >>>> > > > > > > > >> > >> > >> > >>>> > > > > > > > >> > >> >> >> > >>>> > > > > > > > >> > >> Setting up a new cloud environment suitable= for >> > producing >> > >>>> > > > an >> > >>>> > > > > RC may be >> > >>>> > > > > > > > >> > >> time consuming, but you are welcome to try. >> > Krisztian -- >> > >>>> > > > are >> > >>>> > > > > you >> > >>>> > > > > > > > >> > >> available next week to help Micah and >> > potentially take over >> > >>>> > > > > producing >> > >>>> > > > > > > > >> > >> the RC if there are issues? >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > > Sure, I'll be available next week. We can al= so >> > grant access >> > >>>> > > > to >> > >>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow becaus= e >> > configuring >> > >>>> > > > all >> > >>>> > > > > > > > >> > > the CI backends can be time consuming. >> > >>>> > > > > > > > >> > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> > Thanks, >> > >>>> > > > > > > > >> > >> > Micah >> > >>>> > > > > > > > >> > >> > >> > >>>> > > > > > > > >> > >> > [1] >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > >>>> > > > >> > >> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Gui= de >> > >>>> > > > > > > > >> > >> > >> >> > >>>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKin= ney < >> > >>>> > > > > wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> wrote: >> > >>>> > > > > > > > >> > >> >> >> > >>>> > > > > > > > >> > >> >> The process should be well documented at >> > this point but >> > >>>> > > > > there are a >> > >>>> > > > > > > > >> > >> >> number of steps. Note that you need to a= dd >> > your code >> > >>>> > > > > signing key to >> > >>>> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very ha= rd >> > to do). I >> > >>>> > > > > think it's fine >> >> > >>>> > > > > > > > >> > >> >> to hand off the process to others after = the >> > VOTE but it >> > >>>> > > > > would be >> > >>>> > > > > > > > >> > >> >> tricky to have multiple RMs involved wit= h >> > producing the >> > >>>> > > > > source and >> > >>>> > > > > > > > >> > >> >> binary artifacts for the vote >> > >>>> > > > > > > > >> > >> >> >> > >>>> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah >> > Kornfield < >> > >>>> > > > > > > > >> > >> emkornfield@gmail.com> wrote: >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > SGTM, as well. >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > I should have a little bit of time nex= t >> > week if I can >> > >>>> > > > > help as RM but >> > >>>> > > > > > > > >> > >> I have >> > >>>> > > > > > > > >> > >> >> > a couple of concerns: >> > >>>> > > > > > > > >> > >> >> > 1. In the past I've had trouble >> > downloading and >> > >>>> > > > > validating >> > >>>> > > > > > > > >> > >> releases. I'm a >> > >>>> > > > > > > > >> > >> >> > bit worried, that I might have similar >> > problems doing >> > >>>> > > > > the necessary >> > >>>> > > > > > > > >> > >> uploads. >> >> > >>>> > > > > > > > >> > >> >> > 2. My internet connection will likely= be >> > not great, I >> > >>>> > > > > don't know if >> > >>>> > > > > > > > >> > >> this >> > >>>> > > > > > > > >> > >> >> > would make it even less likely to be >> > successful. >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > Does it become problematic if somehow = I >> > would have to >> > >>>> > > > > abandon the >> > >>>> > > > > > > > >> > >> process >> > >>>> > > > > > > > >> > >> >> > mid-release? Is there anyone who coul= d >> > serve as a >> > >>>> > > > > backup? Are the >> > >>>> > > > > > > > >> > >> steps >> > >>>> > > > > > > > >> > >> >> > well documented? >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > Thanks, >> > >>>> > > > > > > > >> > >> >> > Micah >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal >> > Richardson < >> > >>>> > > > > > > > >> > >> neal.p.richardson@gmail.com> >> > >>>> > > > > > > > >> > >> >> > wrote: >> > >>>> > > > > > > > >> > >> >> > >> > >>>> > > > > > > > >> > >> >> > > Sounds good to me. >> > >>>> > > > > > > > >> > >> >> > > >> > >>>> > > > > > > > >> > >> >> > > Do we have a release manager yet? An= y >> > volunteers? >> > >>>> > > > > > > > >> > >> >> > > >> > >>>> > > > > > > > >> > >> >> > > Neal >> > >>>> > > > > > > > >> > >> >> > > >> > >>>> > > > > > > > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes >> > McKinney < >> > >>>> > > > > wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> wrote: >> > >>>> > > > > > > > >> > >> >> > > >> > >>>> > > > > > > > >> > >> >> > > > hi all, >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > > > It looks like we're drawing close = to >> > be able to >> > >>>> > > > > make the 0.15.0 >> > >>>> > > > > > > > >> > >> >> > > > release. I would suggest "pencils >> > down" at the end >> > >>>> > > > > of this week >> > >>>> > > > > > > > >> > >> and >> > >>>> > > > > > > > >> > >> >> > > > see if a release candidate can be >> > produced next >> > >>>> > > > > Monday September >> > >>>> > > > > > > > >> > >> 23. >> > >>>> > > > > > > > >> > >> >> > > > Any thoughts or objections? >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > > > Thanks, >> > >>>> > > > > > > > >> > >> >> > > > Wes >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM W= es >> > McKinney < >> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> >> > > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > hi Eric -- yes, that's correct. = I'm >> > planning to >> > >>>> > > > > amend the >> > >>>> > > > > > > > >> > >> Format docs >> > >>>> > > > > > > > >> > >> >> > > > > today regarding the EOS issue an= d >> > also update >> > >>>> > > > the >> > >>>> > > > > C++ library >> > >>>> > > > > > > > >> > >> >> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM >> > Eric Erhardt >> > >>>> > > > > > > > >> > >> >> > > > > > > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > I assume the plan is to merge = the >> > >>>> > > > > > > > >> > >> ARROW-6313-flatbuffer-alignment >> > >>>> > > > > > > > >> > >> >> > > > branch into master before the 0.15 >> > release, >> > >>>> > > > correct? >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > BTW - I believe the C# alignme= nt >> > changes are >> > >>>> > > > > ready to be >> > >>>> > > > > > > > >> > >> merged into >> > >>>> > > > > > > > >> > >> >> > > > the alignment branch - >> > >>>> > > > > > > > >> > >> https://github.com/apache/arrow/pull/5280/ >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > Eric >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > -----Original Message----- >> > >>>> > > > > > > > >> > >> >> > > > > > From: Micah Kornfield < >> > emkornfield@gmail.com> >> >> > >>>> > > > > > > > >> > >> >> > > > > > Sent: Tuesday, September 10, 2= 019 >> > 10:24 PM >> > >>>> > > > > > > > >> > >> >> > > > > > To: Wes McKinney < >> > wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> >> > > > > > Cc: dev = ; >> > niki.lj < >> > >>>> > > > > niki.lj@aliyun.com> >> > >>>> > > > > > > > >> > >> >> > > > > > Subject: Re: Timeline for 0.15= .0 >> > release >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > I should have a little more >> > bandwidth to help >> > >>>> > > > > with some of >> > >>>> > > > > > > > >> > >> the >> >> > >>>> > > > > > > > >> > >> >> > > > packaging starting tomorrow and go= ing >> > into the >> > >>>> > > > > weekend. >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > On Tuesday, September 10, 2019= , >> > Wes McKinney < >> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> >> > > > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > > Hi folks, >> > >>>> > > > > > > > >> > >> >> > > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > > With the state of nightly >> > packaging and >> > >>>> > > > > integration builds >> > >>>> > > > > > > > >> > >> things >> > >>>> > > > > > > > >> > >> >> > > > > > > aren't looking too good for >> > being in release >> > >>>> > > > > readiness by >> > >>>> > > > > > > > >> > >> the end >> > >>>> > > > > > > > >> > >> >> > > of >> > >>>> > > > > > > > >> > >> >> > > > > > > this week but maybe I'm wron= g. >> > I'm planning >> > >>>> > > > > to be working >> > >>>> > > > > > > > >> > >> to close >> > >>>> > > > > > > > >> > >> >> > > as >> >> > >>>> > > > > > > > >> > >> >> > > > > > > many issues as I can and als= o to >> > help with >> > >>>> > > > > the ongoing >> > >>>> > > > > > > > >> > >> alignment >> > >>>> > > > > > > > >> > >> >> > > > fixes. >> > >>>> > > > > > > > >> > >> >> > > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > > Wes >> > >>>> > > > > > > > >> > >> >> > > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > > On Thu, Sep 5, 2019, 11:07 P= M >> > Micah >> > >>>> > > > Kornfield >> > >>>> > > > > < >> > >>>> > > > > > > > >> > >> >> > > emkornfield@gmail.com >> > >>>> > > > > > > > >> > >> >> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >> Just for reference [1] has = a >> > dashboard of >> > >>>> > > > > the current >> > >>>> > > > > > > > >> > >> issues: >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fcw= i >> > >>>> > > > > > > > >> > >> >> > > > > > >> ki.apache.org >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > se&data=3D02%7C01%7CEric.Erhardt% >> > >>>> > > > > 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034 >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376 >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > 90648216338&sdata=3D0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH= 4 >> > >>>> > > > > > > > >> > >> >> > > > > > >> %3D&reserved=3D0 >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> >> > >>>> > > > > > > > >> > >> >> > > > > > >> On Thu, Sep 5, 2019 at 3:43= PM >> > Wes >> > >>>> > > > McKinney < >> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> >> > > > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> hi all, >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> It doesn't seem like we're >> > going to be in >> > >>>> > > > a >> > >>>> > > > > position to >> > >>>> > > > > > > > >> > >> release >> > >>>> > > > > > > > >> > >> >> > > at >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> the beginning of next week= . I >> > hope that >> > >>>> > > > one >> > >>>> > > > > more week of >> > >>>> > > > > > > > >> > >> work (or >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> less) will be enough to ge= t us >> > there. >> > >>>> > > > Aside >> > >>>> > > > > from merging >> > >>>> > > > > > > > >> > >> the >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> alignment changes, we need= to >> > make sure >> > >>>> > > > > that our >> > >>>> > > > > > > > >> > >> packaging jobs >> > >>>> > > > > > > > >> > >> >> > > > > > >>> required for the release >> > candidate are all >> > >>>> > > > > working. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> If folks could remove issu= es >> > from the >> > >>>> > > > > 0.15.0 backlog >> > >>>> > > > > > > > >> > >> that they >> > >>>> > > > > > > > >> > >> >> > > > don't >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> think they will finish by = end >> > of next week >> > >>>> > > > > that would >> > >>>> > > > > > > > >> > >> help focus >> > >>>> > > > > > > > >> > >> >> > > > > > >>> efforts (there are current= ly >> > 78 issues in >> > >>>> > > > > 0.15.0 still). >> > >>>> > > > > > > > >> > >> I am >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> looking to tackle a few sm= all >> > features >> > >>>> > > > > related to >> > >>>> > > > > > > > >> > >> dictionaries >> > >>>> > > > > > > > >> > >> >> > > > while >> > >>>> > > > > > > > >> > >> >> > > > > > >>> the release window is stil= l >> > open. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> - Wes >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> On Tue, Aug 27, 2019 at 3:= 48 >> > PM Wes >> > >>>> > > > > McKinney < >> > >>>> > > > > > > > >> > >> >> > > wesmckinn@gmail.com> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > hi, >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I think we should try to >> > release the >> > >>>> > > > week >> > >>>> > > > > of September >> > >>>> > > > > > > > >> > >> 9, so >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > development work should = be >> > completed by >> > >>>> > > > > end of next >> > >>>> > > > > > > > >> > >> week. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Does that seem reasonabl= e? >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I plan to get up a patch= for >> > the >> > >>>> > > > protocol >> > >>>> > > > > alignment >> > >>>> > > > > > > > >> > >> changes for >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > C++ in the next couple o= f >> > days -- I >> > >>>> > > > think >> > >>>> > > > > that getting >> > >>>> > > > > > > > >> > >> the >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > alignment work done is t= he >> > main barrier >> > >>>> > > > > to releasing. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Thanks >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Wes >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > On Mon, Aug 19, 2019 at >> > 12:25 PM Ji Liu >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Hi, Wes, on the java s= ide, >> > I can think >> > >>>> > > > > of several >> > >>>> > > > > > > > >> > >> bugs that >> > >>>> > > > > > > > >> > >> >> > > > need >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > to >> > >>>> > > > > > > > >> > >> >> > > > > > >>> be fixed or reminded. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > i. ARROW-6040: Diction= ary >> > entries are >> > >>>> > > > > required in >> > >>>> > > > > > > > >> > >> IPC streams >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > even >> > >>>> > > > > > > > >> > >> >> > > > > > >>> when empty[1] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > This one is under revi= ew >> > now, however >> > >>>> > > > > through this >> > >>>> > > > > > > > >> > >> PR we find >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > that >> > >>>> > > > > > > > >> > >> >> > > > > > >>> there seems a bug in java >> > reading and >> > >>>> > > > > writing >> > >>>> > > > > > > > >> > >> dictionaries in IPC >> > >>>> > > > > > > > >> > >> >> > > > > > >>> which is Inconsistent with >> > spec[2] since >> > >>>> > > > it >> > >>>> > > > > assumes all >> > >>>> > > > > > > > >> > >> >> > > > dictionaries >> > >>>> > > > > > > > >> > >> >> > > > > > >>> are at the start of stream >> > (see details in >> > >>>> > > > > PR comments, >> > >>>> > > > > > > > >> > >> and this >> > >>>> > > > > > > > >> > >> >> > > > > > >>> fix may not catch up with >> > version 0.15). >> > >>>> > > > > @Micah Kornfield >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ii. ARROW-1875: Write >> > 64-bit ints as >> > >>>> > > > > strings in >> > >>>> > > > > > > > >> > >> integration >> > >>>> > > > > > > > >> > >> >> > > > test >> > >>>> > > > > > > > >> > >> >> > > > > > >>> JSON files[3] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Java side code already >> > checked in, >> > >>>> > > > other >> > >>>> > > > > > > > >> > >> implementations >> > >>>> > > > > > > > >> > >> >> > > seems >> > >>>> > > > > > > > >> > >> >> > > > not. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > iii. ARROW-6202: >> > OutOfMemory in >> > >>>> > > > > JdbcAdapter[4] >> > >>>> > > > > > > > >> > >> Caused by >> > >>>> > > > > > > > >> > >> >> > > trying >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > to load all records in= one >> > contiguous >> > >>>> > > > > batch, fixed >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> by providing iterator API = for >> > iteratively >> > >>>> > > > > reading in >> > >>>> > > > > > > > >> > >> >> > > ARROW-6219[5]. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Thanks, >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Ji Liu >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > [1] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F% >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > 2Fgithub.com%2Fapache%2Farrow%2Fpull%2F4960&data=3D02%7C01%7CE >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ric.Erhardt% >> > 40microsoft.com >> >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a4= 5%7 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&a >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > mp;sdata=3DeDF%2FAsJmVs7WjfEuNBYo%2F1TypIN44xx1TTlK6kQHZVg%3D& >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > reserved=3D0 [2] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F% >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Farrow.apache.org >> > >>>> > > > > > > > >> > >> >> > > > >> > %2Fdocs%2Fipc.html&data=3D02%7C01%7CEric.Erh >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ardt%40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > >> > %7Ccbead81a42104034a4f308d736678a45%7C72f988 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdat >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > a=3DH0pM8bVKsOyeORDhHxLlS%2BpaS%2F5meT52wxTKmNssuMk%3D&reserve >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > d=3D0 [3] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F% >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org >> > >>>> > > > > > > > >> > >> >> > > > >> > %2Fjira%2Fbrowse%2FARROW-1875&data=3D02%7C0 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 1%7CEric.Erhardt% >> > 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > 338&sdata=3DcoTpuoEGhfjyOSBTagdlohOTX24DQZmtbWC0gYsDmkM%3D& >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ;reserved=3D0 [4] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F% >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org >> > >>>> > > > > > > > >> > >> >> > > > >> > %2Fjira%2Fbrowse%2FARROW-6202%5B5&data=3D02 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > %7C01%7CEric.Erhardt% >> > 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d73 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > 6678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > 8216338&sdata=3DgnyUMk8cUgwc802QBLF3eAp3mznYwonlbF0qmGyzgmY%3D >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > &reserved=3D0] >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fis >> > >>>> > > > > > > > >> > >> >> > > > > > >>> sues.apache.org >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > %2Fjira%2Fbrowse%2FARROW-6219&data=3D02%7C01%7CEric >> > >>>> > > > > > > > >> > >> >> > > > > > >>> .Erhardt%40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > >> > %7Ccbead81a42104034a4f308d736678a45%7C72f988 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdata=3Dd3 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > >> > LF%2BTeWSprASqO%2ByE4LywlsULHGcb1Iq%2F2byHrEPkY%3D&reserved=3D0 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > ---------------------------------------------------------------- >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > -- From:Wes McKinney < >> > >>>> > > > > wesmckinn@gmail.com> Send >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Time:2019=E5=B9=B48=E6= =9C=8819=E6=97=A5(=E6=98=9F=E6=9C=9F=E4=B8=80) 23:03 >> > To:dev < >> > >>>> > > > > > > > >> > >> dev@arrow.apache.org> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Subject:Re: Timeline f= or >> > 0.15.0 >> > >>>> > > > release >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > I'm going to work some= on >> > organizing >> > >>>> > > > > the 0.15.0 >> > >>>> > > > > > > > >> > >> backlog some >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > this week, if anyone w= ants >> > to help >> > >>>> > > > with >> > >>>> > > > > grooming >> > >>>> > > > > > > > >> > >> >> > > (particularly >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > for languages other th= an >> > C++/Python >> > >>>> > > > > where I'm >> > >>>> > > > > > > > >> > >> focusing) that >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > would be helpful. Ther= e >> > have been >> > >>>> > > > > almost 500 JIRA >> > >>>> > > > > > > > >> > >> issues >> > >>>> > > > > > > > >> > >> >> > > opened >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > since the >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 0.14.0 release, so we >> > should make sure >> > >>>> > > > > to check >> > >>>> > > > > > > > >> > >> whether >> > >>>> > > > > > > > >> > >> >> > > there's >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > any regressions or oth= er >> > serious bugs >> > >>>> > > > > that we should >> > >>>> > > > > > > > >> > >> try to >> > >>>> > > > > > > > >> > >> >> > > fix >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > for 0.15.0. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > On Thu, Aug 15, 2019 a= t >> > 6:23 PM Wes >> > >>>> > > > > McKinney >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The Windows wheel is= sue >> > in 0.14.1 >> > >>>> > > > > seems to be >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > F%2Fissues.apache.or= g >> > >>>> > > > > > > > >> > >> >> > > > >> > %2Fjira%2Fbrowse%2FARROW-6015&data=3D02 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > %7C01%7CEric.Erhardt= % >> > >>>> > > > 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > 736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63= 70376 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > 90648216338&sdata=3DD9lqHR16oRAFlPaIrcXq3UtW%2BLuJQW1= u0Gom2u >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > WEWg0%3D&reserve= d=3D0 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I think the root cau= se >> > could be the >> > >>>> > > > > Windows >> > >>>> > > > > > > > >> > >> changes in >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > F%2Fgithub.com%2Fapache%2Farrow%2Fcommit%2F&data=3D02= %7C01%7 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > CEric.Erhardt% >> > 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > 45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690= 64821 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > 6338&sdata=3DiPmFB%2BncIbmvp5D31vjB4A2KyuMP%2B83Vp7%2= BDiOxvl >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > bs%3D&reserved= =3D0 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > 223ae744cc2a12c60cecb5db593263a03c13f85a >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I would be appreciat= ive >> > if a >> > >>>> > > > > volunteer would look >> > >>>> > > > > > > > >> > >> into what >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > was >> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrong >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > with the 0.14.1 whee= ls >> > on Windows. >> > >>>> > > > > Otherwise >> > >>>> > > > > > > > >> > >> 0.15.0 Windows >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > wheels will be broke= n, >> > too >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The bad wheels can b= e >> > found at >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > >> > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > F%2Fbintray.com%2Fapache%2Farrow%2Fpython%23files%2Fpytho= n%252 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > F0.14.1&data=3D02%7C01%7CEric.Erhardt% >> > >>>> > > > > > > > >> > >> 40microsoft.com >> > >>>> > > > > > > > >> > >> >> > > > %7Ccbea >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > d81a42104034a4f308d736678a45%7C72f988bf86f141af91ab2d7cd0= 11db4 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > > > > >> > >> >> > > > >> >> > >>>> > > > > 7%7C1%7C0%7C637037690648216338&sdata=3DvZzx4HNS9qp2UW= hFagqfJ >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> > >>>> > > > > zbY%2BGzwspH1TO3wdfrbA6Y%3D&reserved=3D0 >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > On Thu, Aug 15, 2019= at >> > 1:28 PM >> > >>>> > > > > Antoine Pitrou < >> > >>>> > > > > > > > >> > >> >> > > > > > >>> solipsis@pitrou.net> wrote= : >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > On Thu, 15 Aug 201= 9 >> > 11:17:07 -0700 >> > >>>> > > > > Micah >> > >>>> > > > > > > > >> > >> Kornfield >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > < >> emkornfield@gmail.com> >> > wrote: >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > In C++ they ar= e >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > independent, w= e >> > could have >> > >>>> > > > > 32-bit array >> > >>>> > > > > > > > >> > >> lengths and >> > >>>> > > > > > > > >> > >> >> > > > > > >>> variable-length >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > types with 64-= bit >> > offsets if >> > >>>> > > > we >> > >>>> > > > > wanted (we >> > >>>> > > > > > > > >> > >> just >> > >>>> > > > > > > > >> > >> >> > > > wouldn't >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > be >> > >>>> > > > > > > > >> > >> >> > > > > > >>> able to >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > have a List ch= ild >> > with more >> > >>>> > > > > than INT32_MAX >> > >>>> > > > > > > > >> > >> elements). >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > I think the poin= t is >> > we could do >> > >>>> > > > > this in C++ >> > >>>> > > > > > > > >> > >> but we >> > >>>> > > > > > > > >> > >> >> > > > don't. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> I'm not sure we >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > would have >> > introduced the >> > >>>> > > > "Large" >> > >>>> > > > > types if we >> > >>>> > > > > > > > >> > >> did. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 64-bit offsets tak= e >> > twice as much >> > >>>> > > > > space as 32-bit >> > >>>> > > > > > > > >> > >> >> > > offsets, >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > so if >> > >>>> > > > > > > > >> > >> >> > > > > > >>> you're >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > storing lots of >> > small-ish lists or >> > >>>> > > > > strings, >> > >>>> > > > > > > > >> > >> 32-bit >> > >>>> > > > > > > > >> > >> >> > > offsets >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > are preferrable. = So >> > even with >> > >>>> > > > > 64-bit array >> > >>>> > > > > > > > >> > >> lengths from >> > >>>> > > > > > > > >> > >> >> > > > the >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > start >> > >>>> > > > > > > > >> > >> >> > > > > > >>> it would >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > still be beneficia= l to >> > have types >> > >>>> > > > > with 32-bit >> > >>>> > > > > > > > >> > >> offsets. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > Going with the >> > limited address >> > >>>> > > > > space in Java >> > >>>> > > > > > > > >> > >> and >> > >>>> > > > > > > > >> > >> >> > > calling >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > it a >> > >>>> > > > > > > > >> > >> >> > > > > > >>> reference >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > implementation s= eems >> > suboptimal. >> > >>>> > > > > If a consumer >> > >>>> > > > > > > > >> > >> uses a >> > >>>> > > > > > > > >> > >> >> > > > "Large" >> > >>>> > > > > > > > >> > >> >> > > > > > >>> type >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > presumably it is >> > because they >> > >>>> > > > > need the ability >> > >>>> > > > > > > > >> > >> to store >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > more >> > >>>> > > > > > > > >> > >> >> > > > > > >>> than INT32_MAX >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > child elements i= n a >> > column, >> > >>>> > > > > otherwise it is >> > >>>> > > > > > > > >> > >> just >> > >>>> > > > > > > > >> > >> >> > > wasting >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > space >> > >>>> > > > > > > > >> > >> >> > > > > > >>> [1]. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Probably. Though i= f >> > the individual >> > >>>> > > > > elements >> > >>>> > > > > > > > >> > >> (lists or >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > strings) >> > >>>> > > > > > > > >> > >> >> > > > > > >>> are >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > large, not much sp= ace >> > is wasted in >> > >>>> > > > > proportion, >> > >>>> > > > > > > > >> > >> so it may >> > >>>> > > > > > > > >> > >> >> > > be >> > >>>> > > > > > > > >> > >> >> > > > > > >>> simpler in >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > such a case to alw= ays >> > create a >> > >>>> > > > > "Large" type >> > >>>> > > > > > > > >> > >> array. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > [1] I suppose >> > theoretically >> > >>>> > > > there >> > >>>> > > > > might be some >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > performance >> > >>>> > > > > > > > >> > >> >> > > > > > >>> benefits on >> >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > 64-bit architect= ures >> > to using >> > >>>> > > > the >> > >>>> > > > > native word >> > >>>> > > > > > > > >> > >> sizes. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Concretely, common >> > 64-bit >> > >>>> > > > > architectures don't do >> > >>>> > > > > > > > >> > >> that, as >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 32-bit >> > >>>> > > > > > > > >> > >> >> > > > > > >>> is an >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > extremely common >> > integer size even >> > >>>> > > > > in >> > >>>> > > > > > > > >> > >> high-performance >> > >>>> > > > > > > > >> > >> >> > > > code. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Regards >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Antoine. >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >> > >>>> > > > > > > > >> > >> >> > > > > > >>> >> > >>>> > > > > > > > >> > >> >> > > > > > >> >> > >>>> > > > > > > > >> > >> >> > > > >> > >>>> > > > > > > > >> > >> >> > > >> > >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > > >> > >>>> > > > > >> > >>>> > > > >> > >> >> >> --0000000000006d325e0593755a1f--