arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kouhei Sutou <...@clear-code.com>
Subject Re: Timeline for 0.13 Arrow release
Date Wed, 27 Mar 2019 20:57:50 GMT
Hi,

> What controls do you think we can put in place to prevent the issues
> you had with RC0-2? For example, I merged a patch that affected the
> dev/release/02-source.sh script, in the future I could test that out
> (though perhaps we should develop an automated way to test these
> scripts)

  * RC0 problem: We can't build conda package on Windows:
    https://github.com/apache/arrow/pull/4042

    * This is caused by
      https://github.com/apache/arrow/pull/3832 and
      https://github.com/apache/arrow/pull/4024 .

    * If we rebased
      https://github.com/apache/arrow/pull/4024 on master
      and test it before we merge, we could notice the
      problem.

    * We could notice the problem by nightly package build.

    * Conclusion: We should finish packaging changes on the
      previous day of the release day.

  * RC1 problem: The above fix isn't enough. :<
    https://github.com/apache/arrow/pull/4043

    * I could notice the problem by testings all package
      builds. I tested only wheel builds.

    * We could notice the problem by nightly package build.

    * Conclusion: We should finish packaging changes on the
      previous day of the release day.

  * RC2 problem: I used the commit that doesn't include the
    fix for the RC1 problem. :<

    * I forgot to merge the upstream master into the local
      master before I create RC. It's a human error...


Thanks,
--
kou

In <CAJPUwMBZd6pLZffOUVLFVuAyHtLPEGFUr3waw1Lj75OG9JLiXA@mail.gmail.com>
  "Re: Timeline for 0.13 Arrow release" on Wed, 27 Mar 2019 07:29:45 -0500,
  Wes McKinney <wesmckinn@gmail.com> wrote:

> Thanks Kou for being the RM!
> 
> What controls do you think we can put in place to prevent the issues
> you had with RC0-2? For example, I merged a patch that affected the
> dev/release/02-source.sh script, in the future I could test that out
> (though perhaps we should develop an automated way to test these
> scripts)
> 
> On Wed, Mar 27, 2019 at 2:43 AM Kouhei Sutou <kou@clear-code.com> wrote:
>>
>> Hi,
>>
>> I'm creating RC3(!).
>> There are some problems in RC0-2. I hope that we can vote
>> against RC3.
>>
>> We need to wait building and uploading packages:
>>
>>   https://github.com/kou/crossbow/branches/all?query=build-71
>>
>>
>> I'll be able to send vote e-mail in 12 hours...
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CAOrRV3cY+=0b+kodO_VhYq4HU8jQvz5Bp7QJOBju+wA7t13T0g@mail.gmail.com>
>>   "Re: Timeline for 0.13 Arrow release" on Tue, 26 Mar 2019 12:51:05 -0700,
>>   Javier Luraschi <javier@rstudio.com> wrote:
>>
>> > Great, thank you, Wes!
>> >
>> > On Tue, Mar 26, 2019 at 12:45 PM Wes McKinney <wesmckinn@gmail.com> wrote:
>> >
>> >> Javier -- it was merged using our usual merge tool.
>> >>
>> >> On Tue, Mar 26, 2019, 8:38 PM Javier Luraschi <javier@rstudio.com>
wrote:
>> >>
>> >> > https://github.com/apache/arrow/pull/4011 got closed but is needed
for
>> >> > 0.13
>> >> > for the R package, could someone reopen please? We can wait for Romains
>> >> to
>> >> > give a LGTM and then merge. Thank you!
>> >> >
>> >> > On Tue, Mar 26, 2019 at 12:38 PM Kouhei Sutou <kou@clear-code.com>
>> >> wrote:
>> >> >
>> >> > > Hi,
>> >> > >
>> >> > > We don't freeze anything.
>> >> > > We can merge pull requests when they are ready to merge as usual.
>> >> > >
>> >> > >
>> >> > > Thanks,
>> >> > > --
>> >> > > kou
>> >> > >
>> >> > > In <6531c437-6bf9-61cd-c7c6-7be06fa96516@python.org>
>> >> > >   "Re: Timeline for 0.13 Arrow release" on Tue, 26 Mar 2019 14:58:06
>> >> > +0100,
>> >> > >   Antoine Pitrou <antoine@python.org> wrote:
>> >> > >
>> >> > > >
>> >> > > > Hi Kou,
>> >> > > >
>> >> > > > What should be the policy for merges until 0.13.0 is released?
>> >> > > > Do you want to instate a feature freeze or commit freeze?
>> >> > > >
>> >> > > > Regards
>> >> > > >
>> >> > > > Antoine.
>> >> > > >
>> >> > > >
>> >> > > > Le 19/03/2019 à 15:46, Kouhei Sutou a écrit :
>> >> > > >> Hi,
>> >> > > >>
>> >> > > >> There are no blockers on GLib, Ruby and Linux packages.
>> >> > > >>
>> >> > > >> Can we include JavaScript into 0.13.0?
>> >> > > >> If we include JavaScript into 0.13.0, we can remove
>> >> > > >> codes to release JavaScript separately. For example,
we can
>> >> > > >> remove dev/release/js-*. We can enable version update
code
>> >> > > >> in dev/release/00-prepare.sh:
>> >> > > >>
>> >> > >
>> >> >
>> >> https://github.com/apache/arrow/blob/master/dev/release/00-prepare.sh#L67-L74
>> >> > > >>
>> >> > > >> We can merge "JavaScript Releases" document into our
release
>> >> > > >> document:
>> >> > > >>
>> >> > >
>> >> >
>> >> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-JavaScriptReleases
>> >> > > >>
>> >> > > >>
>> >> > > >> Thanks,
>> >> > > >> --
>> >> > > >> kou
>> >> > > >>
>> >> > > >> In <
>> >> > CAJPUwMBGjzbwrwYBwse6BD9LnN_7XOZN_aQ2JOB9_MPVMHc07w@mail.gmail.com>
>> >> > > >>   "Re: Timeline for 0.13 Arrow release" on Mon, 18 Mar
2019 20:51:12
>> >> > > -0500,
>> >> > > >>   Wes McKinney <wesmckinn@gmail.com> wrote:
>> >> > > >>
>> >> > > >>> hi folks,
>> >> > > >>>
>> >> > > >>> I think we're basically at the 0.13 end game here.
There's some
>> >> more
>> >> > > >>> patches can get in, but do we all think we can cut
an RC by the end
>> >> > of
>> >> > > >>> the week? What are the blocking issues?
>> >> > > >>>
>> >> > > >>> Thanks
>> >> > > >>> Wes
>> >> > > >>>
>> >> > > >>> On Sat, Mar 16, 2019 at 9:57 PM Kouhei Sutou <kou@clear-code.com>
>> >> > > wrote:
>> >> > > >>>>
>> >> > > >>>> Hi,
>> >> > > >>>>
>> >> > > >>>>> Submitted the packaging builds:
>> >> > > >>>>>
>> >> > >
>> >> >
>> >> https://github.com/kszucs/crossbow/branches/all?utf8=%E2%9C%93&query=build-452
>> >> > > >>>>
>> >> > > >>>> I've fixed .deb/.rpm packages:
>> >> > > https://github.com/apache/arrow/pull/3934
>> >> > > >>>> It has been merged.
>> >> > > >>>> So .deb/.rpm packages are ready for release.
>> >> > > >>>>
>> >> > > >>>> Thanks,
>> >> > > >>>> --
>> >> > > >>>> kou
>> >> > > >>>>
>> >> > > >>>> In <
>> >> > > CAHM19a5SOmZxGcphc6eE-Mr2USVVhWB252UdGjrVocQ-CB2dag@mail.gmail.com>
>> >> > > >>>>   "Re: Timeline for 0.13 Arrow release" on Thu,
14 Mar 2019
>> >> 16:24:43
>> >> > > +0100,
>> >> > > >>>>   Krisztián Szűcs <szucs.krisztian@gmail.com>
wrote:
>> >> > > >>>>
>> >> > > >>>>> Submitted the packaging builds:
>> >> > > >>>>>
>> >> > >
>> >> >
>> >> https://github.com/kszucs/crossbow/branches/all?utf8=%E2%9C%93&query=build-452
>> >> > > >>>>>
>> >> > > >>>>> On Thu, Mar 14, 2019 at 4:19 PM Wes McKinney
<
>> >> wesmckinn@gmail.com>
>> >> > > wrote:
>> >> > > >>>>>
>> >> > > >>>>>> The CMake refactor is merged! Kudos to
Uwe for 3+ weeks of hard
>> >> > > labor on
>> >> > > >>>>>> this.
>> >> > > >>>>>>
>> >> > > >>>>>> We should run all the packaging tasks
and get a full accounting
>> >> of
>> >> > > >>>>>> what is broken so we aren't surprised
during the release process
>> >> > > >>>>>>
>> >> > > >>>>>> On Wed, Mar 13, 2019 at 9:39 AM Krisztián
Szűcs
>> >> > > >>>>>> <szucs.krisztian@gmail.com> wrote:
>> >> > > >>>>>>>
>> >> > > >>>>>>> The proof of the pudding is in the
eating. You convinced me.
>> >> > > >>>>>>>
>> >> > > >>>>>>> On Wed, Mar 13, 2019 at 3:31 PM Wes
McKinney <
>> >> > wesmckinn@gmail.com>
>> >> > > >>>>>> wrote:
>> >> > > >>>>>>>
>> >> > > >>>>>>>> Krisztian -- are you all right
with proceeding with merging
>> >> the
>> >> > > CMake
>> >> > > >>>>>>>> refactor? I'm pretty committed
to helping fix the problems
>> >> that
>> >> > > come
>> >> > > >>>>>>>> up. Since most consumers of the
project don't test until
>> >> > _after_ a
>> >> > > >>>>>>>> release, we won't find out about
some problems until we merge
>> >> it
>> >> > > and
>> >> > > >>>>>>>> release it. Thus, IMHO it doesn't
make sense to wait another
>> >> > 8-10
>> >> > > >>>>>>>> weeks since we'd be delaying
feedback for that long. There are
>> >> > > also a
>> >> > > >>>>>>>> number of follow-on issues blocking
on the refactor
>> >> > > >>>>>>>>
>> >> > > >>>>>>>> On Tue, Mar 12, 2019 at 11:39
AM Andy Grove <
>> >> > > andygrove73@gmail.com>
>> >> > > >>>>>> wrote:
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> I've cleaned up my issues
for Rust, moving most of them to
>> >> > > 0.14.0.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> I have two PRs in progress
that I would appreciate reviews
>> >> on:
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> https://github.com/apache/arrow/pull/3671
- [Rust] Table API
>> >> > > (a.k.a
>> >> > > >>>>>>>>> DataFrame)
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> https://github.com/apache/arrow/pull/3851
- [Rust] Parquet
>> >> > data
>> >> > > >>>>>> source
>> >> > > >>>>>>>> in
>> >> > > >>>>>>>>> DataFusion
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> Once these are merged I have
some small follow up PRs for
>> >> > 0.13.0
>> >> > > >>>>>> that I
>> >> > > >>>>>>>> can
>> >> > > >>>>>>>>> get done this week.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> Thanks,
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> Andy.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> On Tue, Mar 12, 2019 at 8:21
AM Wes McKinney <
>> >> > > wesmckinn@gmail.com>
>> >> > > >>>>>>>> wrote:
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>>> hi folks,
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> I think we are on track
to be able to release toward the end
>> >> > of
>> >> > > >>>>>> this
>> >> > > >>>>>>>>>> month. My proposed timeline:
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> * This week (March 11-15):
feature/improvement push mostly
>> >> > > >>>>>>>>>> * Next week (March 18-22):
shift to bug fixes,
>> >> stabilization,
>> >> > > empty
>> >> > > >>>>>>>>>> backlog of feature/improvement
JIRAs
>> >> > > >>>>>>>>>> * Week of March 25: propose
release candidate
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> Does this seem reasonable?
This puts us at about 9-10 weeks
>> >> > from
>> >> > > >>>>>> 0.12.
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> We need an RM for 0.13,
any PMCs want to volunteer?
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> Take a look at our release
page:
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>
>> >> > > >>>>>>
>> >> > >
>> >> >
>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103091219
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> Out of the open or in-progress
issues, we have:
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> * C#: 3 issues
>> >> > > >>>>>>>>>> * C++ (all components):
51 issues
>> >> > > >>>>>>>>>> * Java: 3 issues
>> >> > > >>>>>>>>>> * Python: 38 issues
>> >> > > >>>>>>>>>> * Rust (all components):
33 issues
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> Please help curating
the backlogs for each component.
>> >> There's
>> >> > a
>> >> > > >>>>>>>>>> smattering of issues
in other categories. There are also 10
>> >> > open
>> >> > > >>>>>>>>>> issues with No Component
(and 20 resolved issues), those
>> >> need
>> >> > > their
>> >> > > >>>>>>>>>> metadata fixed.
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> Thanks,
>> >> > > >>>>>>>>>> Wes
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>>> On Wed, Feb 27, 2019
at 1:49 PM Wes McKinney <
>> >> > > wesmckinn@gmail.com>
>> >> > > >>>>>>>> wrote:
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> The timeline for
the 0.13 release is drawing closer. I
>> >> would
>> >> > > say
>> >> > > >>>>>> we
>> >> > > >>>>>>>>>>> should consider a
release candidate either the week of
>> >> March
>> >> > 18
>> >> > > >>>>>> or
>> >> > > >>>>>>>>>>> March 25, which gives
us ~3 weeks to close out backlog
>> >> items.
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> There are around
220 issues open or in-progress in
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>
>> >> > > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.13.0+Release
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> Please have a look.
If issues are not assigned to someone
>> >> as
>> >> > > the
>> >> > > >>>>>> next
>> >> > > >>>>>>>>>>> couple of weeks pass
by I'll begin moving at least C++ and
>> >> > > Python
>> >> > > >>>>>>>>>>> issues to 0.14 that
don't seem like they're going to get
>> >> done
>> >> > > for
>> >> > > >>>>>>>>>>> 0.13. If development
stakeholders for C#, Java, Rust, Ruby,
>> >> > and
>> >> > > >>>>>> other
>> >> > > >>>>>>>>>>> components can review
and curate the issues that would be
>> >> > > >>>>>> helpful.
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> You can help keep
the JIRA issues tidy by making sure to
>> >> add
>> >> > > Fix
>> >> > > >>>>>>>>>>> Version to issues
and to make sure to add a Component so
>> >> that
>> >> > > >>>>>> issues
>> >> > > >>>>>>>>>>> are properly categorized
in the release notes.
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> Thanks
>> >> > > >>>>>>>>>>> Wes
>> >> > > >>>>>>>>>>>
>> >> > > >>>>>>>>>>> On Sat, Feb 9, 2019
at 10:39 AM Wes McKinney <
>> >> > > >>>>>> wesmckinn@gmail.com>
>> >> > > >>>>>>>>>> wrote:
>> >> > > >>>>>>>>>>>>
>> >> > > >>>>>>>>>>>> See
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>
>> >> > > >>>>>>
>> >> > >
>> >> >
>> >> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >> > > >>>>>>>>>>>>
>> >> > > >>>>>>>>>>>> The source release
step is one of the places where
>> >> problems
>> >> > > >>>>>> occur.
>> >> > > >>>>>>>>>>>>
>> >> > > >>>>>>>>>>>> On Sat, Feb 9,
2019, 10:33 AM <aneesh@quiltdata.io wrote:
>> >> > > >>>>>>>>>>>>>
>> >> > > >>>>>>>>>>>>>
>> >> > > >>>>>>>>>>>>>> On Feb
8, 2019, at 9:19 AM, Uwe L. Korn <
>> >> mail@uwekorn.com
>> >> > >
>> >> > > >>>>>>>> wrote:
>> >> > > >>>>>>>>>>>>>>
>> >> > > >>>>>>>>>>>>>> We could
dockerize some of the release steps to ensure
>> >> > that
>> >> > > >>>>>> they
>> >> > > >>>>>>>>>> run in the same environment.
>> >> > > >>>>>>>>>>>>>
>> >> > > >>>>>>>>>>>>> I may be
able to help with said Dockerization. If not for
>> >> > > this
>> >> > > >>>>>>>>>> release, then for the
next. Are there docs on which systems
>> >> we
>> >> > > >>>>>> wish to
>> >> > > >>>>>>>>>> target and/or any build
steps beyond the current dev
>> >> > container (
>> >> > > >>>>>>>>>> https://github.com/apache/arrow/tree/master/dev/container)?
>> >> > > >>>>>>>>>>
>> >> > > >>>>>>>>
>> >> > > >>>>>>
>> >> > >
>> >> >
>> >>
Mime
View raw message