arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krisztián Szűcs <szucs.kriszt...@gmail.com>
Subject Re: Timeline for 0.15.0 release
Date Thu, 26 Sep 2019 14:11:48 GMT
There are still missing linux artifacts [1]:
- for amd64 debug symbol packages
- for arm64 optional CUDA, plasma and Gandiva modules

I think we can safely ignore them for the release, crossbow will
report them as missing but the artifact downloading step finish.

Let me know Micah if you have any issues.

[1]: https://github.com/apache/arrow/pull/5506#issuecomment-535495351

On Thu, Sep 26, 2019 at 3:38 PM Micah Kornfield <emkornfield@gmail.com>
wrote:

> Yes, I merged it and it will be included.  I needed to start over due to a
> cross-bow issue...
>
> On Thu, Sep 26, 2019 at 7:18 AM Ji Liu <niki.lj@aliyun.com> wrote:
>
>> Hi Micah,
>> Hmm, unfortunately, I just found a bug in JDBC adapter and open a
>> PR, could this change catch up with 0.15?
>> See https://github.com/apache/arrow/pull/5511
>>
>>
>> Thanks,
>> Ji Liu
>>
>>
>> ------------------------------------------------------------------
>> From:Micah Kornfield <emkornfield@gmail.com>
>> Send Time:2019年9月26日(星期四) 14:23
>> To:Neal Richardson <neal.p.richardson@gmail.com>
>> Cc:"Krisztián Szűcs" <szucs.krisztian@gmail.com>; Wes McKinney <
>> wesmckinn@gmail.com>; dev <dev@arrow.apache.org>
>> Subject:Re: Timeline for 0.15.0 release
>>
>> Just an I've started the RC generation process off, the last commit from
>> master is [1]
>>
>> I am currently waiting the crossbow builds (build-690 on
>> ursa-labs/crossbow).  I think this will take a little while so I will pick
>> it up tomorrow (Thursday).
>>
>> Thanks,
>> Micah
>>
>> [1]
>>
>> https://github.com/apache/arrow/commit/07ab5083d5a2925ced6f8168b60b8fa336f4eccc
>>
>> On Wed, Sep 25, 2019 at 2:07 PM Neal Richardson <
>> neal.p.richardson@gmail.com>
>> wrote:
>>
>> > IMO it's too risky to add something that adds a dependency
>> > (aws-sdk-cpp) on the day of cutting a release.
>> >
>> > Neal
>> >
>> > On Wed, Sep 25, 2019 at 12:54 PM Krisztián Szűcs
>> > <szucs.krisztian@gmail.com> wrote:
>> > >
>> > > We don't have a comprehensive documentation yet, so let's postpone it.
>> > >
>> > >
>> > > On Wed, Sep 25, 2019 at 9:48 PM Krisztián Szűcs <
>> > szucs.krisztian@gmail.com> wrote:
>> > >>
>> > >> The S3 python bindings would be a nice addition to the release.
>> > >> I don't think we should block on this but the PR is ready. Opinions?
>> > >> https://github.com/apache/arrow/pull/5423
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On Wed, Sep 25, 2019 at 5:28 PM Micah Kornfield <
>> emkornfield@gmail.com>
>> > wrote:
>> > >>>
>> > >>> OK, I'll start the process today.  I'll send up e-mail updates as I
>> > make progress.
>> > >>>
>> > >>> On Wed, Sep 25, 2019 at 8:22 AM Wes McKinney <wesmckinn@gmail.com>
>> > wrote:
>> > >>>>
>> > >>>> Yes, all systems go as far as I'm concerned.
>> > >>>>
>> > >>>> On Wed, Sep 25, 2019 at 9:56 AM Neal Richardson
>> > >>>> <neal.p.richardson@gmail.com> wrote:
>> > >>>> >
>>
>> > >>>> > Andy's DataFusion issue and Wes's Parquet one have both been merged,
>>
>> > >>>> > and it looks like the LICENSE issue is being resolved as I type. So
>> > >>>> > are we good to go now?
>> > >>>> >
>> > >>>> > Neal
>> > >>>> >
>> > >>>> >
>> > >>>> > On Tue, Sep 24, 2019 at 10:30 PM Andy Grove <
>> andygrove73@gmail.com>
>> > wrote:
>> > >>>> > >
>> > >>>> > > I found a last minute issue with DataFusion (Rust) and would
>> > appreciate it
>> > >>>> > > if we could merge ARROW-6086 (PR is
>> > >>>> > > https://github.com/apache/arrow/pull/5494
>> ) before cutting the RC.
>> > >>>> > >
>> > >>>> > > Thanks,
>> > >>>> > >
>> > >>>> > > Andy.
>> > >>>> > >
>> > >>>> > >
>> > >>>> > > On Tue, Sep 24, 2019 at 6:19 PM Micah Kornfield <
>> > emkornfield@gmail.com>
>> > >>>> > > wrote:
>> > >>>> > >
>> > >>>> > > > OK, I'm going to postpone cutting a release until tomorrow
>> > (hoping we can
>> > >>>> > > > issues resolved by then)..  I'll also try to review the
>> > third-party
>> > >>>> > > > additions since 14.x.
>> > >>>> > > >
>> > >>>> > > > On Tue, Sep 24, 2019 at 4:20 PM Wes McKinney <
>> > wesmckinn@gmail.com> wrote:
>> > >>>> > > >
>> > >>>> > > > > I found a licensing issue
>> > >>>> > > > >
>> > >>>> > > > > https://issues.apache.org/jira/browse/ARROW-6679
>> > >>>> > > > >
>> > >>>> > > > > It might be worth examining third party code added to the
>> > project
>> > >>>> > > > > since 0.14.x to make sure there are no other such issues.
>> > >>>> > > > >
>> > >>>> > > > > On Tue, Sep 24, 2019 at 6:10 PM Wes McKinney <
>> > wesmckinn@gmail.com>
>> > >>>> > > > wrote:
>> > >>>> > > > > >
>>
>> > >>>> > > > > > I have diagnosed the problem (Thrift "string" data must be
>> > UTF-8,
>>
>> > >>>> > > > > > cannot be arbitrary binary) and am working on a patch right
>> > now
>> > >>>> > > > > >
>> > >>>> > > > > > On Tue, Sep 24, 2019 at 6:02 PM Wes McKinney <
>> > wesmckinn@gmail.com>
>> > >>>> > > > > wrote:
>> > >>>> > > > > > >
>> > >>>> > > > > > > I just opened
>> > >>>> > > > > > >
>> > >>>> > > > > > > https://issues.apache.org/jira/browse/ARROW-6678
>> > >>>> > > > > > >
>> > >>>> > > > > > > Please don't cut an RC until I have an opportunity to
>> > diagnose this,
>> > >>>> > > > > > > will report back.
>> > >>>> > > > > > >
>> > >>>> > > > > > >
>> > >>>> > > > > > > On Tue, Sep 24, 2019 at 5:51 PM Wes McKinney <
>> > wesmckinn@gmail.com>
>> > >>>> > > > > wrote:
>> > >>>> > > > > > > >
>> > >>>> > > > > > > > I'm investigating a possible Parquet-related
>> > compatibility bug
>> > >>>> > > > that I
>> > >>>> > > > > > > > encountered through some routine testing /
>> > benchmarking. I'll
>> > >>>> > > > report
>> > >>>> > > > > > > > back once I figure out what is going on (if anything)
>> > >>>> > > > > > > >
>> > >>>> > > > > > > > On Sun, Sep 22, 2019 at 11:51 PM Micah Kornfield <
>> > >>>> > > > > emkornfield@gmail.com> wrote:
>> > >>>> > > > > > > > >>
>> > >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trust
>> > (i.e. you can
>> > >>>> > > > > get it
>> > >>>> > > > > > > > >> signed by another PMC member), but is not 100%
>> > essential.
>> > >>>> > > > > > > > >
>> > >>>> > > > > > > > > That won't be an option for me this week (it seems
>> > like I would
>> > >>>> > > > > need to meet one face-to-face).  I'll try to get the GPG
>> > checked in and
>> > >>>> > > > the
>> > >>>> > > > > rest of the pre-requisites done tomorrow (Monday) to
>> > hopefully start the
>> > >>>> > > > > release on Tuesday (hopefully we can solve the last
>> > blocker/integration
>> > >>>> > > > > tests by then).
>> > >>>> > > > > > > > >
>> > >>>> > > > > > > > > On Sat, Sep 21, 2019 at 7:12 PM Wes McKinney <
>> > >>>> > > > wesmckinn@gmail.com>
>> > >>>> > > > > wrote:
>> > >>>> > > > > > > > >>
>> > >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trust
>> > (i.e. you can
>> > >>>> > > > > get it
>> > >>>> > > > > > > > >> signed by another PMC member), but is not 100%
>> > essential.
>> > >>>> > > > > > > > >>
>> > >>>> > > > > > > > >> Speaking of the release, there are at least 2 code
>> > changes I
>> > >>>> > > > still
>> > >>>> > > > > > > > >> want to get in
>> > >>>> > > > > > > > >>
>> > >>>> > > > > > > > >> ARROW-5717
>> > >>>> > > > > > > > >> ARROW-6353
>> > >>>> > > > > > > > >>
>>
>> > >>>> > > > > > > > >> I just pushed updates to ARROW-5717, will merge once
>> > the build
>> > >>>> > > > is
>> > >>>> > > > > green.
>> > >>>> > > > > > > > >>
>>
>> > >>>> > > > > > > > >> There are a couple of Rust patches still marked for
>> > 0.15. The
>> > >>>> > > > rest
>> > >>>> > > > > > > > >> seems to be documentation and a couple of
>> > integration test
>> > >>>> > > > > failures we
>> > >>>> > > > > > > > >> should see about fixing in time.
>> > >>>> > > > > > > > >>
>> > >>>> > > > > > > > >> On Fri, Sep 20, 2019 at 11:26 PM Micah Kornfield <
>> > >>>> > > > > emkornfield@gmail.com> wrote:
>> > >>>> > > > > > > > >> >
>> > >>>> > > > > > > > >> > Thanks Krisztián and Wes,
>>
>> > >>>> > > > > > > > >> > I've gone ahead and started registering myself on
>> > all the
>> > >>>> > > > > packaging sites.
>> > >>>> > > > > > > > >> >
>>
>> > >>>> > > > > > > > >> > Is there any review process when adding my GPG key
>> > to the SVN
>> > >>>> > > > > file? [1]
>> > >>>> > > > > > > > >> > doesn't seem to mention explicitly.
>> > >>>> > > > > > > > >> >
>> > >>>> > > > > > > > >> > Thanks,
>> > >>>> > > > > > > > >> > Micah
>> > >>>> > > > > > > > >> >
>> > >>>> > > > > > > > >> > [1]
>> > https://www.apache.org/dev/version-control.html#https-svn
>> > >>>> > > > > > > > >> >
>>
>> > >>>> > > > > > > > >> > On Fri, Sep 20, 2019 at 5:01 PM Krisztián Szűcs <
>> > >>>> > > > > szucs.krisztian@gmail.com>
>> > >>>> > > > > > > > >> > wrote:
>> > >>>> > > > > > > > >> >
>> > >>>> > > > > > > > >> > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney <
>> > >>>> > > > > wesmckinn@gmail.com> wrote:
>> > >>>> > > > > > > > >> > >
>> > >>>> > > > > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah
>> > Kornfield <
>> > >>>> > > > > emkornfield@gmail.com>
>> > >>>> > > > > > > > >> > >> wrote:
>> > >>>> > > > > > > > >> > >> >>
>> > >>>> > > > > > > > >> > >> >> The process should be well documented at
>> > this point but
>> > >>>> > > > > there are a
>> > >>>> > > > > > > > >> > >> >> number of steps.
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for the
>> > release?
>> > >>>> > > >  Are
>> > >>>> > > > > there
>> > >>>> > > > > > > > >> > >> instructions for the adding the code signing
>> > Key to SVN?
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > I will make a go of it.  i will try to
>> > mitigate any
>> > >>>> > > > > internet issues by
>> > >>>> > > > > > > > >> > >> doing the process for a cloud instance (I
>> > assume that isn't
>> > >>>> > > > > a problem?).
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >>
>>
>> > >>>> > > > > > > > >> > >> Setting up a new cloud environment suitable for
>> > producing
>> > >>>> > > > an
>> > >>>> > > > > RC may be
>> > >>>> > > > > > > > >> > >> time consuming, but you are welcome to try.
>> > Krisztian --
>> > >>>> > > > are
>> > >>>> > > > > you
>> > >>>> > > > > > > > >> > >> available next week to help Micah and
>> > potentially take over
>> > >>>> > > > > producing
>> > >>>> > > > > > > > >> > >> the RC if there are issues?
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > > > > > >> > > Sure, I'll be available next week. We can also
>> > grant access
>> > >>>> > > > to
>> > >>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because
>> > configuring
>> > >>>> > > > all
>> > >>>> > > > > > > > >> > > the CI backends can be time consuming.
>> > >>>> > > > > > > > >> > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > > > > > >> > >> > Thanks,
>> > >>>> > > > > > > > >> > >> > Micah
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > [1]
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > >>>> > > >
>> >
>> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> > >>>> > > > > > > > >> > >> >
>>
>> > >>>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
>> > >>>> > > > > wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> wrote:
>> > >>>> > > > > > > > >> > >> >>
>> > >>>> > > > > > > > >> > >> >> The process should be well documented at
>> > this point but
>> > >>>> > > > > there are a
>> > >>>> > > > > > > > >> > >> >> number of steps. Note that you need to add
>> > your code
>> > >>>> > > > > signing key to
>> > >>>> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard
>> > to do). I
>> > >>>> > > > > think it's fine
>>
>> > >>>> > > > > > > > >> > >> >> to hand off the process to others after the
>> > VOTE but it
>> > >>>> > > > > would be
>> > >>>> > > > > > > > >> > >> >> tricky to have multiple RMs involved with
>> > producing the
>> > >>>> > > > > source and
>> > >>>> > > > > > > > >> > >> >> binary artifacts for the vote
>> > >>>> > > > > > > > >> > >> >>
>> > >>>> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah
>> > Kornfield <
>> > >>>> > > > > > > > >> > >> emkornfield@gmail.com> wrote:
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > SGTM, as well.
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > I should have a little bit of time next
>> > week if I can
>> > >>>> > > > > help as RM but
>> > >>>> > > > > > > > >> > >> I have
>> > >>>> > > > > > > > >> > >> >> > a couple of concerns:
>> > >>>> > > > > > > > >> > >> >> > 1.  In the past I've had trouble
>> > downloading and
>> > >>>> > > > > validating
>> > >>>> > > > > > > > >> > >> releases. I'm a
>> > >>>> > > > > > > > >> > >> >> > bit worried, that I might have similar
>> > problems doing
>> > >>>> > > > > the necessary
>> > >>>> > > > > > > > >> > >> uploads.
>>
>> > >>>> > > > > > > > >> > >> >> > 2.  My internet connection will likely be
>> > not great, I
>> > >>>> > > > > don't know if
>> > >>>> > > > > > > > >> > >> this
>> > >>>> > > > > > > > >> > >> >> > would make it even less likely to be
>> > successful.
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > Does it become problematic if somehow I
>> > would have to
>> > >>>> > > > > abandon the
>> > >>>> > > > > > > > >> > >> process
>> > >>>> > > > > > > > >> > >> >> > mid-release?  Is there anyone who could
>> > serve as a
>> > >>>> > > > > backup?  Are the
>> > >>>> > > > > > > > >> > >> steps
>> > >>>> > > > > > > > >> > >> >> > well documented?
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > Thanks,
>> > >>>> > > > > > > > >> > >> >> > Micah
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal
>> > Richardson <
>> > >>>> > > > > > > > >> > >> neal.p.richardson@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > wrote:
>> > >>>> > > > > > > > >> > >> >> >
>> > >>>> > > > > > > > >> > >> >> > > Sounds good to me.
>> > >>>> > > > > > > > >> > >> >> > >
>> > >>>> > > > > > > > >> > >> >> > > Do we have a release manager yet? Any
>> > volunteers?
>> > >>>> > > > > > > > >> > >> >> > >
>> > >>>> > > > > > > > >> > >> >> > > Neal
>> > >>>> > > > > > > > >> > >> >> > >
>> > >>>> > > > > > > > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes
>> > McKinney <
>> > >>>> > > > > wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> wrote:
>> > >>>> > > > > > > > >> > >> >> > >
>> > >>>> > > > > > > > >> > >> >> > > > hi all,
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >> >> > > > It looks like we're drawing close to
>> > be able to
>> > >>>> > > > > make the 0.15.0
>> > >>>> > > > > > > > >> > >> >> > > > release. I would suggest "pencils
>> > down" at the end
>> > >>>> > > > > of this week
>> > >>>> > > > > > > > >> > >> and
>> > >>>> > > > > > > > >> > >> >> > > > see if a release candidate can be
>> > produced next
>> > >>>> > > > > Monday September
>> > >>>> > > > > > > > >> > >> 23.
>> > >>>> > > > > > > > >> > >> >> > > > Any thoughts or objections?
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >> >> > > > Thanks,
>> > >>>> > > > > > > > >> > >> >> > > > Wes
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes
>> > McKinney <
>> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > hi Eric -- yes, that's correct. I'm
>> > planning to
>> > >>>> > > > > amend the
>> > >>>> > > > > > > > >> > >> Format docs
>> > >>>> > > > > > > > >> > >> >> > > > > today regarding the EOS issue and
>> > also update
>> > >>>> > > > the
>> > >>>> > > > > C++ library
>> > >>>> > > > > > > > >> > >> >> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM
>> > Eric Erhardt
>> > >>>> > > > > > > > >> > >> >> > > > > <Eric.Erhardt@microsoft.com
>> > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > I assume the plan is to merge the
>> > >>>> > > > > > > > >> > >> ARROW-6313-flatbuffer-alignment
>> > >>>> > > > > > > > >> > >> >> > > > branch into master before the 0.15
>> > release,
>> > >>>> > > > correct?
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > BTW - I believe the C# alignment
>> > changes are
>> > >>>> > > > > ready to be
>> > >>>> > > > > > > > >> > >> merged into
>> > >>>> > > > > > > > >> > >> >> > > > the alignment branch -
>> > >>>> > > > > > > > >> > >> https://github.com/apache/arrow/pull/5280/
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > Eric
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > -----Original Message-----
>> > >>>> > > > > > > > >> > >> >> > > > > > From: Micah Kornfield <
>> > emkornfield@gmail.com>
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > Sent: Tuesday, September 10, 2019
>> > 10:24 PM
>> > >>>> > > > > > > > >> > >> >> > > > > > To: Wes McKinney <
>> > wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > > > > Cc: dev <dev@arrow.apache.org>;
>> > niki.lj <
>> > >>>> > > > > niki.lj@aliyun.com>
>> > >>>> > > > > > > > >> > >> >> > > > > > Subject: Re: Timeline for 0.15.0
>> > release
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > I should have a little more
>> > bandwidth to help
>> > >>>> > > > > with some of
>> > >>>> > > > > > > > >> > >> the
>>
>> > >>>> > > > > > > > >> > >> >> > > > packaging starting tomorrow and going
>> > into the
>> > >>>> > > > > weekend.
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > On Tuesday, September 10, 2019,
>> > Wes McKinney <
>> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > > Hi folks,
>> > >>>> > > > > > > > >> > >> >> > > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > > With the state of nightly
>> > packaging and
>> > >>>> > > > > integration builds
>> > >>>> > > > > > > > >> > >> things
>> > >>>> > > > > > > > >> > >> >> > > > > > > aren't looking too good for
>> > being in release
>> > >>>> > > > > readiness by
>> > >>>> > > > > > > > >> > >> the end
>> > >>>> > > > > > > > >> > >> >> > > of
>> > >>>> > > > > > > > >> > >> >> > > > > > > this week but maybe I'm wrong.
>> > I'm planning
>> > >>>> > > > > to be working
>> > >>>> > > > > > > > >> > >> to close
>> > >>>> > > > > > > > >> > >> >> > > as
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > > many issues as I can and also to
>> > help with
>> > >>>> > > > > the ongoing
>> > >>>> > > > > > > > >> > >> alignment
>> > >>>> > > > > > > > >> > >> >> > > > fixes.
>> > >>>> > > > > > > > >> > >> >> > > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > > Wes
>> > >>>> > > > > > > > >> > >> >> > > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > > On Thu, Sep 5, 2019, 11:07 PM
>> > Micah
>> > >>>> > > > Kornfield
>> > >>>> > > > > <
>> > >>>> > > > > > > > >> > >> >> > > emkornfield@gmail.com
>> > >>>> > > > > > > > >> > >> >> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >> Just for reference [1] has a
>> > dashboard of
>> > >>>> > > > > the current
>> > >>>> > > > > > > > >> > >> issues:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
>> > >>>> > > > > > > > >> > >> >> > > > > > >> ki.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > se&amp;data=02%7C01%7CEric.Erhardt%
>> > >>>> > > > > 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > 90648216338&amp;sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
>> > >>>> > > > > > > > >> > >> >> > > > > > >> %3D&amp;reserved=0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >> On Thu, Sep 5, 2019 at 3:43 PM
>> > Wes
>> > >>>> > > > McKinney <
>> > >>>> > > > > > > > >> > >> wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> hi all,
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> It doesn't seem like we're
>> > going to be in
>> > >>>> > > > a
>> > >>>> > > > > position to
>> > >>>> > > > > > > > >> > >> release
>> > >>>> > > > > > > > >> > >> >> > > at
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> the beginning of next week. I
>> > hope that
>> > >>>> > > > one
>> > >>>> > > > > more week of
>> > >>>> > > > > > > > >> > >> work (or
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> less) will be enough to get us
>> > there.
>> > >>>> > > > Aside
>> > >>>> > > > > from merging
>> > >>>> > > > > > > > >> > >> the
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> alignment changes, we need to
>> > make sure
>> > >>>> > > > > that our
>> > >>>> > > > > > > > >> > >> packaging jobs
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> required for the release
>> > candidate are all
>> > >>>> > > > > working.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> If folks could remove issues
>> > from the
>> > >>>> > > > > 0.15.0 backlog
>> > >>>> > > > > > > > >> > >> that they
>> > >>>> > > > > > > > >> > >> >> > > > don't
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> think they will finish by end
>> > of next week
>> > >>>> > > > > that would
>> > >>>> > > > > > > > >> > >> help focus
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> efforts (there are currently
>> > 78 issues in
>> > >>>> > > > > 0.15.0 still).
>> > >>>> > > > > > > > >> > >> I am
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> looking to tackle a few small
>> > features
>> > >>>> > > > > related to
>> > >>>> > > > > > > > >> > >> dictionaries
>> > >>>> > > > > > > > >> > >> >> > > > while
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> the release window is still
>> > open.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> - Wes
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> On Tue, Aug 27, 2019 at 3:48
>> > PM Wes
>> > >>>> > > > > McKinney <
>> > >>>> > > > > > > > >> > >> >> > > wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > hi,
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I think we should try to
>> > release the
>> > >>>> > > > week
>> > >>>> > > > > of September
>> > >>>> > > > > > > > >> > >> 9, so
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > development work should be
>> > completed by
>> > >>>> > > > > end of next
>> > >>>> > > > > > > > >> > >> week.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Does that seem reasonable?
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I plan to get up a patch for
>> > the
>> > >>>> > > > protocol
>> > >>>> > > > > alignment
>> > >>>> > > > > > > > >> > >> changes for
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > C++ in the next couple of
>> > days -- I
>> > >>>> > > > think
>> > >>>> > > > > that getting
>> > >>>> > > > > > > > >> > >> the
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > alignment work done is the
>> > main barrier
>> > >>>> > > > > to releasing.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Thanks
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Wes
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > On Mon, Aug 19, 2019 at
>> > 12:25 PM Ji Liu
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > <niki.lj@aliyun.com.invalid
>> >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Hi, Wes, on the java side,
>> > I can think
>> > >>>> > > > > of several
>> > >>>> > > > > > > > >> > >> bugs that
>> > >>>> > > > > > > > >> > >> >> > > > need
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > to
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> be fixed or reminded.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > i. ARROW-6040: Dictionary
>> > entries are
>> > >>>> > > > > required in
>> > >>>> > > > > > > > >> > >> IPC streams
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > even
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> when empty[1]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > This one is under review
>> > now, however
>> > >>>> > > > > through this
>> > >>>> > > > > > > > >> > >> PR we find
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > that
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> there seems a bug in java
>> > reading and
>> > >>>> > > > > writing
>> > >>>> > > > > > > > >> > >> dictionaries in IPC
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> which is Inconsistent with
>> > spec[2] since
>> > >>>> > > > it
>> > >>>> > > > > assumes all
>> > >>>> > > > > > > > >> > >> >> > > > dictionaries
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> are at the start of stream
>> > (see details in
>> > >>>> > > > > PR comments,
>> > >>>> > > > > > > > >> > >> and this
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> fix may not catch up with
>> > version 0.15).
>> > >>>> > > > > @Micah Kornfield
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ii. ARROW-1875: Write
>> > 64-bit ints as
>> > >>>> > > > > strings in
>> > >>>> > > > > > > > >> > >> integration
>> > >>>> > > > > > > > >> > >> >> > > > test
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> JSON files[3]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Java side code already
>> > checked in,
>> > >>>> > > > other
>> > >>>> > > > > > > > >> > >> implementations
>> > >>>> > > > > > > > >> > >> >> > > seems
>> > >>>> > > > > > > > >> > >> >> > > > not.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > iii. ARROW-6202:
>> > OutOfMemory in
>> > >>>> > > > > JdbcAdapter[4]
>> > >>>> > > > > > > > >> > >> Caused by
>> > >>>> > > > > > > > >> > >> >> > > trying
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > to load all records in one
>> > contiguous
>> > >>>> > > > > batch, fixed
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> by providing iterator API for
>> > iteratively
>> > >>>> > > > > reading in
>> > >>>> > > > > > > > >> > >> >> > > ARROW-6219[5].
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Thanks,
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Ji Liu
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > [1]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > 2Fgithub.com%2Fapache%2Farrow%2Fpull%2F4960&amp;data=02%7C01%7CE
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ric.Erhardt%
>> > 40microsoft.com
>>
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a45%7
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&a
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > mp;sdata=eDF%2FAsJmVs7WjfEuNBYo%2F1TypIN44xx1TTlK6kQHZVg%3D&amp;
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > reserved=0 [2]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Farrow.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %2Fdocs%2Fipc.html&amp;data=02%7C01%7CEric.Erh
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ardt%40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %7Ccbead81a42104034a4f308d736678a45%7C72f988
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&amp;sdat
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > a=H0pM8bVKsOyeORDhHxLlS%2BpaS%2F5meT52wxTKmNssuMk%3D&amp;reserve
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > d=0 [3]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %2Fjira%2Fbrowse%2FARROW-1875&amp;data=02%7C0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 1%7CEric.Erhardt%
>> > 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > 338&amp;sdata=coTpuoEGhfjyOSBTagdlohOTX24DQZmtbWC0gYsDmkM%3D&amp
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ;reserved=0 [4]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %2Fjira%2Fbrowse%2FARROW-6202%5B5&amp;data=02
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > %7C01%7CEric.Erhardt%
>> > 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d73
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > 6678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > 8216338&amp;sdata=gnyUMk8cUgwc802QBLF3eAp3mznYwonlbF0qmGyzgmY%3D
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > &amp;reserved=0]
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fis
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> sues.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > %2Fjira%2Fbrowse%2FARROW-6219&amp;data=02%7C01%7CEric
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> .Erhardt%40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %7Ccbead81a42104034a4f308d736678a45%7C72f988
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&amp;sdata=d3
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > LF%2BTeWSprASqO%2ByE4LywlsULHGcb1Iq%2F2byHrEPkY%3D&amp;reserved=0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > ----------------------------------------------------------------
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > -- From:Wes McKinney <
>> > >>>> > > > > wesmckinn@gmail.com> Send
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Time:2019年8月19日(星期一) 23:03
>> > To:dev <
>> > >>>> > > > > > > > >> > >> dev@arrow.apache.org>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Subject:Re: Timeline for
>> > 0.15.0
>> > >>>> > > > release
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > I'm going to work some on
>> > organizing
>> > >>>> > > > > the 0.15.0
>> > >>>> > > > > > > > >> > >> backlog some
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > this week, if anyone wants
>> > to help
>> > >>>> > > > with
>> > >>>> > > > > grooming
>> > >>>> > > > > > > > >> > >> >> > > (particularly
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > for languages other than
>> > C++/Python
>> > >>>> > > > > where I'm
>> > >>>> > > > > > > > >> > >> focusing) that
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > would be helpful. There
>> > have been
>> > >>>> > > > > almost 500 JIRA
>> > >>>> > > > > > > > >> > >> issues
>> > >>>> > > > > > > > >> > >> >> > > opened
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > since the
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > 0.14.0 release, so we
>> > should make sure
>> > >>>> > > > > to check
>> > >>>> > > > > > > > >> > >> whether
>> > >>>> > > > > > > > >> > >> >> > > there's
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > any regressions or other
>> > serious bugs
>> > >>>> > > > > that we should
>> > >>>> > > > > > > > >> > >> try to
>> > >>>> > > > > > > > >> > >> >> > > fix
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > for 0.15.0.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > On Thu, Aug 15, 2019 at
>> > 6:23 PM Wes
>> > >>>> > > > > McKinney
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > <wesmckinn@gmail.com>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The Windows wheel issue
>> > in 0.14.1
>> > >>>> > > > > seems to be
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > F%2Fissues.apache.org
>> > >>>> > > > > > > > >> > >> >> > > >
>> > %2Fjira%2Fbrowse%2FARROW-6015&amp;data=02
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > %7C01%7CEric.Erhardt%
>> > >>>> > > > 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > 736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > 90648216338&amp;sdata=D9lqHR16oRAFlPaIrcXq3UtW%2BLuJQW1u0Gom2u
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > WEWg0%3D&amp;reserved=0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I think the root cause
>> > could be the
>> > >>>> > > > > Windows
>> > >>>> > > > > > > > >> > >> changes in
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > F%2Fgithub.com%2Fapache%2Farrow%2Fcommit%2F&amp;data=02%7C01%7
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > CEric.Erhardt%
>> > 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > 45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064821
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > 6338&amp;sdata=iPmFB%2BncIbmvp5D31vjB4A2KyuMP%2B83Vp7%2BDiOxvl
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > bs%3D&amp;reserved=0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > 223ae744cc2a12c60cecb5db593263a03c13f85a
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I would be appreciative
>> > if a
>> > >>>> > > > > volunteer would look
>> > >>>> > > > > > > > >> > >> into what
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > was
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> wrong
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > with the 0.14.1 wheels
>> > on Windows.
>> > >>>> > > > > Otherwise
>> > >>>> > > > > > > > >> > >> 0.15.0 Windows
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > wheels will be broken,
>> > too
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The bad wheels can be
>> > found at
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > >
>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > F%2Fbintray.com%2Fapache%2Farrow%2Fpython%23files%2Fpython%252
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > F0.14.1&amp;data=02%7C01%7CEric.Erhardt%
>> > >>>> > > > > > > > >> > >> 40microsoft.com
>> > >>>> > > > > > > > >> > >> >> > > > %7Ccbea
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > d81a42104034a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db4
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > > > > >> > >> >> > > >
>>
>> > >>>> > > > > 7%7C1%7C0%7C637037690648216338&amp;sdata=vZzx4HNS9qp2UWhFagqfJ
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>> > >>>> > > > > zbY%2BGzwspH1TO3wdfrbA6Y%3D&amp;reserved=0
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > On Thu, Aug 15, 2019 at
>> > 1:28 PM
>> > >>>> > > > > Antoine Pitrou <
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> solipsis@pitrou.net> wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > On Thu, 15 Aug 2019
>> > 11:17:07 -0700
>> > >>>> > > > > Micah
>> > >>>> > > > > > > > >> > >> Kornfield
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > <
>> emkornfield@gmail.com>
>> > wrote:
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > In C++ they are
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > independent, we
>> > could have
>> > >>>> > > > > 32-bit array
>> > >>>> > > > > > > > >> > >> lengths and
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> variable-length
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > types with 64-bit
>> > offsets if
>> > >>>> > > > we
>> > >>>> > > > > wanted (we
>> > >>>> > > > > > > > >> > >> just
>> > >>>> > > > > > > > >> > >> >> > > > wouldn't
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > be
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> able to
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > have a List child
>> > with more
>> > >>>> > > > > than INT32_MAX
>> > >>>> > > > > > > > >> > >> elements).
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > I think the point is
>> > we could do
>> > >>>> > > > > this in C++
>> > >>>> > > > > > > > >> > >> but we
>> > >>>> > > > > > > > >> > >> >> > > > don't.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> I'm not sure we
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > would have
>> > introduced the
>> > >>>> > > > "Large"
>> > >>>> > > > > types if we
>> > >>>> > > > > > > > >> > >> did.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 64-bit offsets take
>> > twice as much
>> > >>>> > > > > space as 32-bit
>> > >>>> > > > > > > > >> > >> >> > > offsets,
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > so if
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> you're
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > storing lots of
>> > small-ish lists or
>> > >>>> > > > > strings,
>> > >>>> > > > > > > > >> > >> 32-bit
>> > >>>> > > > > > > > >> > >> >> > > offsets
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > are preferrable.  So
>> > even with
>> > >>>> > > > > 64-bit array
>> > >>>> > > > > > > > >> > >> lengths from
>> > >>>> > > > > > > > >> > >> >> > > > the
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > start
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> it would
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > still be beneficial to
>> > have types
>> > >>>> > > > > with 32-bit
>> > >>>> > > > > > > > >> > >> offsets.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > Going with the
>> > limited address
>> > >>>> > > > > space in Java
>> > >>>> > > > > > > > >> > >> and
>> > >>>> > > > > > > > >> > >> >> > > calling
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > it a
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> reference
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > implementation seems
>> > suboptimal.
>> > >>>> > > > > If a consumer
>> > >>>> > > > > > > > >> > >> uses a
>> > >>>> > > > > > > > >> > >> >> > > > "Large"
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> type
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > presumably it is
>> > because they
>> > >>>> > > > > need the ability
>> > >>>> > > > > > > > >> > >> to store
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > more
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> than INT32_MAX
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > child elements in a
>> > column,
>> > >>>> > > > > otherwise it is
>> > >>>> > > > > > > > >> > >> just
>> > >>>> > > > > > > > >> > >> >> > > wasting
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > space
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> [1].
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Probably. Though if
>> > the individual
>> > >>>> > > > > elements
>> > >>>> > > > > > > > >> > >> (lists or
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > strings)
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> are
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > large, not much space
>> > is wasted in
>> > >>>> > > > > proportion,
>> > >>>> > > > > > > > >> > >> so it may
>> > >>>> > > > > > > > >> > >> >> > > be
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> simpler in
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > such a case to always
>> > create a
>> > >>>> > > > > "Large" type
>> > >>>> > > > > > > > >> > >> array.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > [1] I suppose
>> > theoretically
>> > >>>> > > > there
>> > >>>> > > > > might be some
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > performance
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> benefits on
>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > 64-bit architectures
>> > to using
>> > >>>> > > > the
>> > >>>> > > > > native word
>> > >>>> > > > > > > > >> > >> sizes.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Concretely, common
>> > 64-bit
>> > >>>> > > > > architectures don't do
>> > >>>> > > > > > > > >> > >> that, as
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 32-bit
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> is an
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > extremely common
>> > integer size even
>> > >>>> > > > > in
>> > >>>> > > > > > > > >> > >> high-performance
>> > >>>> > > > > > > > >> > >> >> > > > code.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Regards
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Antoine.
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > >
>> > >>>> > > > > > > > >> > >> >> > > > > > >>>
>> > >>>> > > > > > > > >> > >> >> > > > > > >>
>> > >>>> > > > > > > > >> > >> >> > > >
>> > >>>> > > > > > > > >> > >> >> > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > > > > > >> > >
>> > >>>> > > > >
>> > >>>> > > >
>> >
>>
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message