arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kouhei Sutou <...@clear-code.com>
Subject Re: Timeline for 0.13 Arrow release
Date Wed, 27 Mar 2019 21:00:47 GMT
Hi,

> Technically such a bugfix in 02-source.sh does not require
> invalidating an RC because the produced artifacts are unaffected. But
> it's definitely a rough edge in the RM process and best avoided

Yes. I found some problems for release scripts:

  * https://github.com/apache/arrow/pull/4040
  * https://github.com/apache/arrow/pull/4041

but I fixed them locally and continued RC creation. I sent
these pull requests when I found the RC0 has a problem.


Thanks,
--
kou

In <CAJPUwMAbZGuwgFSf2W-QkosbLTz4Oovi-vWvgzNw5kjrA-R1+Q@mail.gmail.com>
  "Re: Timeline for 0.13 Arrow release" on Wed, 27 Mar 2019 07:42:42 -0500,
  Wes McKinney <wesmckinn@gmail.com> wrote:

> Technically such a bugfix in 02-source.sh does not require
> invalidating an RC because the produced artifacts are unaffected. But
> it's definitely a rough edge in the RM process and best avoided
> 
> On Wed, Mar 27, 2019 at 7:38 AM Francois Saint-Jacques
> <fsaintjacques@gmail.com> wrote:
>>
>> An automated nightly release is for sure going to catch a lot of the tiny
>> issues. Regarding the error in the script, do you need a full RC increase
>> if one modifies the script, can you just modify it locally?
>>
>> François
>>
>> On Wed, Mar 27, 2019 at 8:30 AM Wes McKinney <wesmckinn@gmail.com> wrote:
>>
>> > Thanks Kou for being the RM!
>> >
>> > What controls do you think we can put in place to prevent the issues
>> > you had with RC0-2? For example, I merged a patch that affected the
>> > dev/release/02-source.sh script, in the future I could test that out
>> > (though perhaps we should develop an automated way to test these
>> > scripts)
>> >
>> > On Wed, Mar 27, 2019 at 2:43 AM Kouhei Sutou <kou@clear-code.com> wrote:
>> > >
>> > > Hi,
>> > >
>> > > I'm creating RC3(!).
>> > > There are some problems in RC0-2. I hope that we can vote
>> > > against RC3.
>> > >
>> > > We need to wait building and uploading packages:
>> > >
>> > >   https://github.com/kou/crossbow/branches/all?query=build-71
>> > >
>> > >
>> > > I'll be able to send vote e-mail in 12 hours...
>> > >
>> > >
>> > > Thanks,
>> > > --
>> > > kou
>> > >
>> > > In <CAOrRV3cY+=0b+kodO_VhYq4HU8jQvz5Bp7QJOBju+wA7t13T0g@mail.gmail.com>
>> > >   "Re: Timeline for 0.13 Arrow release" on Tue, 26 Mar 2019 12:51:05
>> > -0700,
>> > >   Javier Luraschi <javier@rstudio.com> wrote:
>> > >
>> > > > Great, thank you, Wes!
>> > > >
>> > > > On Tue, Mar 26, 2019 at 12:45 PM Wes McKinney <wesmckinn@gmail.com>
>> > wrote:
>> > > >
>> > > >> Javier -- it was merged using our usual merge tool.
>> > > >>
>> > > >> On Tue, Mar 26, 2019, 8:38 PM Javier Luraschi <javier@rstudio.com>
>> > wrote:
>> > > >>
>> > > >> > https://github.com/apache/arrow/pull/4011 got closed but
is needed
>> > for
>> > > >> > 0.13
>> > > >> > for the R package, could someone reopen please? We can wait
for
>> > Romains
>> > > >> to
>> > > >> > give a LGTM and then merge. Thank you!
>> > > >> >
>> > > >> > On Tue, Mar 26, 2019 at 12:38 PM Kouhei Sutou <kou@clear-code.com>
>> > > >> wrote:
>> > > >> >
>> > > >> > > Hi,
>> > > >> > >
>> > > >> > > We don't freeze anything.
>> > > >> > > We can merge pull requests when they are ready to merge
as usual.
>> > > >> > >
>> > > >> > >
>> > > >> > > Thanks,
>> > > >> > > --
>> > > >> > > kou
>> > > >> > >
>> > > >> > > In <6531c437-6bf9-61cd-c7c6-7be06fa96516@python.org>
>> > > >> > >   "Re: Timeline for 0.13 Arrow release" on Tue, 26 Mar
2019
>> > 14:58:06
>> > > >> > +0100,
>> > > >> > >   Antoine Pitrou <antoine@python.org> wrote:
>> > > >> > >
>> > > >> > > >
>> > > >> > > > Hi Kou,
>> > > >> > > >
>> > > >> > > > What should be the policy for merges until 0.13.0
is released?
>> > > >> > > > Do you want to instate a feature freeze or commit
freeze?
>> > > >> > > >
>> > > >> > > > Regards
>> > > >> > > >
>> > > >> > > > Antoine.
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > Le 19/03/2019 à 15:46, Kouhei Sutou a écrit :
>> > > >> > > >> Hi,
>> > > >> > > >>
>> > > >> > > >> There are no blockers on GLib, Ruby and Linux
packages.
>> > > >> > > >>
>> > > >> > > >> Can we include JavaScript into 0.13.0?
>> > > >> > > >> If we include JavaScript into 0.13.0, we can
remove
>> > > >> > > >> codes to release JavaScript separately. For
example, we can
>> > > >> > > >> remove dev/release/js-*. We can enable version
update code
>> > > >> > > >> in dev/release/00-prepare.sh:
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://github.com/apache/arrow/blob/master/dev/release/00-prepare.sh#L67-L74
>> > > >> > > >>
>> > > >> > > >> We can merge "JavaScript Releases" document
into our release
>> > > >> > > >> document:
>> > > >> > > >>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-JavaScriptReleases
>> > > >> > > >>
>> > > >> > > >>
>> > > >> > > >> Thanks,
>> > > >> > > >> --
>> > > >> > > >> kou
>> > > >> > > >>
>> > > >> > > >> In <
>> > > >> > CAJPUwMBGjzbwrwYBwse6BD9LnN_7XOZN_aQ2JOB9_MPVMHc07w@mail.gmail.com>
>> > > >> > > >>   "Re: Timeline for 0.13 Arrow release" on
Mon, 18 Mar 2019
>> > 20:51:12
>> > > >> > > -0500,
>> > > >> > > >>   Wes McKinney <wesmckinn@gmail.com>
wrote:
>> > > >> > > >>
>> > > >> > > >>> hi folks,
>> > > >> > > >>>
>> > > >> > > >>> I think we're basically at the 0.13 end
game here. There's
>> > some
>> > > >> more
>> > > >> > > >>> patches can get in, but do we all think
we can cut an RC by
>> > the end
>> > > >> > of
>> > > >> > > >>> the week? What are the blocking issues?
>> > > >> > > >>>
>> > > >> > > >>> Thanks
>> > > >> > > >>> Wes
>> > > >> > > >>>
>> > > >> > > >>> On Sat, Mar 16, 2019 at 9:57 PM Kouhei
Sutou <
>> > kou@clear-code.com>
>> > > >> > > wrote:
>> > > >> > > >>>>
>> > > >> > > >>>> Hi,
>> > > >> > > >>>>
>> > > >> > > >>>>> Submitted the packaging builds:
>> > > >> > > >>>>>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://github.com/kszucs/crossbow/branches/all?utf8=%E2%9C%93&query=build-452
>> > > >> > > >>>>
>> > > >> > > >>>> I've fixed .deb/.rpm packages:
>> > > >> > > https://github.com/apache/arrow/pull/3934
>> > > >> > > >>>> It has been merged.
>> > > >> > > >>>> So .deb/.rpm packages are ready for
release.
>> > > >> > > >>>>
>> > > >> > > >>>> Thanks,
>> > > >> > > >>>> --
>> > > >> > > >>>> kou
>> > > >> > > >>>>
>> > > >> > > >>>> In <
>> > > >> > >
>> > CAHM19a5SOmZxGcphc6eE-Mr2USVVhWB252UdGjrVocQ-CB2dag@mail.gmail.com>
>> > > >> > > >>>>   "Re: Timeline for 0.13 Arrow release"
on Thu, 14 Mar 2019
>> > > >> 16:24:43
>> > > >> > > +0100,
>> > > >> > > >>>>   Krisztián Szűcs <szucs.krisztian@gmail.com>
wrote:
>> > > >> > > >>>>
>> > > >> > > >>>>> Submitted the packaging builds:
>> > > >> > > >>>>>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://github.com/kszucs/crossbow/branches/all?utf8=%E2%9C%93&query=build-452
>> > > >> > > >>>>>
>> > > >> > > >>>>> On Thu, Mar 14, 2019 at 4:19 PM
Wes McKinney <
>> > > >> wesmckinn@gmail.com>
>> > > >> > > wrote:
>> > > >> > > >>>>>
>> > > >> > > >>>>>> The CMake refactor is merged!
Kudos to Uwe for 3+ weeks of
>> > hard
>> > > >> > > labor on
>> > > >> > > >>>>>> this.
>> > > >> > > >>>>>>
>> > > >> > > >>>>>> We should run all the packaging
tasks and get a full
>> > accounting
>> > > >> of
>> > > >> > > >>>>>> what is broken so we aren't
surprised during the release
>> > process
>> > > >> > > >>>>>>
>> > > >> > > >>>>>> On Wed, Mar 13, 2019 at 9:39
AM Krisztián Szűcs
>> > > >> > > >>>>>> <szucs.krisztian@gmail.com>
wrote:
>> > > >> > > >>>>>>>
>> > > >> > > >>>>>>> The proof of the pudding
is in the eating. You convinced
>> > me.
>> > > >> > > >>>>>>>
>> > > >> > > >>>>>>> On Wed, Mar 13, 2019 at
3:31 PM Wes McKinney <
>> > > >> > wesmckinn@gmail.com>
>> > > >> > > >>>>>> wrote:
>> > > >> > > >>>>>>>
>> > > >> > > >>>>>>>> Krisztian -- are you
all right with proceeding with
>> > merging
>> > > >> the
>> > > >> > > CMake
>> > > >> > > >>>>>>>> refactor? I'm pretty
committed to helping fix the
>> > problems
>> > > >> that
>> > > >> > > come
>> > > >> > > >>>>>>>> up. Since most consumers
of the project don't test until
>> > > >> > _after_ a
>> > > >> > > >>>>>>>> release, we won't find
out about some problems until we
>> > merge
>> > > >> it
>> > > >> > > and
>> > > >> > > >>>>>>>> release it. Thus, IMHO
it doesn't make sense to wait
>> > another
>> > > >> > 8-10
>> > > >> > > >>>>>>>> weeks since we'd be
delaying feedback for that long.
>> > There are
>> > > >> > > also a
>> > > >> > > >>>>>>>> number of follow-on
issues blocking on the refactor
>> > > >> > > >>>>>>>>
>> > > >> > > >>>>>>>> On Tue, Mar 12, 2019
at 11:39 AM Andy Grove <
>> > > >> > > andygrove73@gmail.com>
>> > > >> > > >>>>>> wrote:
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> I've cleaned up
my issues for Rust, moving most of them
>> > to
>> > > >> > > 0.14.0.
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> I have two PRs
in progress that I would appreciate
>> > reviews
>> > > >> on:
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> https://github.com/apache/arrow/pull/3671
- [Rust]
>> > Table API
>> > > >> > > (a.k.a
>> > > >> > > >>>>>>>>> DataFrame)
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> https://github.com/apache/arrow/pull/3851
- [Rust]
>> > Parquet
>> > > >> > data
>> > > >> > > >>>>>> source
>> > > >> > > >>>>>>>> in
>> > > >> > > >>>>>>>>> DataFusion
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> Once these are
merged I have some small follow up PRs
>> > for
>> > > >> > 0.13.0
>> > > >> > > >>>>>> that I
>> > > >> > > >>>>>>>> can
>> > > >> > > >>>>>>>>> get done this week.
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> Thanks,
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> Andy.
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>> On Tue, Mar 12,
2019 at 8:21 AM Wes McKinney <
>> > > >> > > wesmckinn@gmail.com>
>> > > >> > > >>>>>>>> wrote:
>> > > >> > > >>>>>>>>>
>> > > >> > > >>>>>>>>>> hi folks,
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> I think we
are on track to be able to release toward
>> > the end
>> > > >> > of
>> > > >> > > >>>>>> this
>> > > >> > > >>>>>>>>>> month. My proposed
timeline:
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> * This week
(March 11-15): feature/improvement push
>> > mostly
>> > > >> > > >>>>>>>>>> * Next week
(March 18-22): shift to bug fixes,
>> > > >> stabilization,
>> > > >> > > empty
>> > > >> > > >>>>>>>>>> backlog of
feature/improvement JIRAs
>> > > >> > > >>>>>>>>>> * Week of March
25: propose release candidate
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> Does this seem
reasonable? This puts us at about 9-10
>> > weeks
>> > > >> > from
>> > > >> > > >>>>>> 0.12.
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> We need an
RM for 0.13, any PMCs want to volunteer?
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> Take a look
at our release page:
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>
>> > > >> > > >>>>>>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103091219
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> Out of the
open or in-progress issues, we have:
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> * C#: 3 issues
>> > > >> > > >>>>>>>>>> * C++ (all
components): 51 issues
>> > > >> > > >>>>>>>>>> * Java: 3 issues
>> > > >> > > >>>>>>>>>> * Python: 38
issues
>> > > >> > > >>>>>>>>>> * Rust (all
components): 33 issues
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> Please help
curating the backlogs for each component.
>> > > >> There's
>> > > >> > a
>> > > >> > > >>>>>>>>>> smattering
of issues in other categories. There are
>> > also 10
>> > > >> > open
>> > > >> > > >>>>>>>>>> issues with
No Component (and 20 resolved issues),
>> > those
>> > > >> need
>> > > >> > > their
>> > > >> > > >>>>>>>>>> metadata fixed.
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> Thanks,
>> > > >> > > >>>>>>>>>> Wes
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>>> On Wed, Feb
27, 2019 at 1:49 PM Wes McKinney <
>> > > >> > > wesmckinn@gmail.com>
>> > > >> > > >>>>>>>> wrote:
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> The timeline
for the 0.13 release is drawing closer. I
>> > > >> would
>> > > >> > > say
>> > > >> > > >>>>>> we
>> > > >> > > >>>>>>>>>>> should
consider a release candidate either the week of
>> > > >> March
>> > > >> > 18
>> > > >> > > >>>>>> or
>> > > >> > > >>>>>>>>>>> March 25,
which gives us ~3 weeks to close out backlog
>> > > >> items.
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> There are
around 220 issues open or in-progress in
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>
>> > > >> > >
>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.13.0+Release
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> Please
have a look. If issues are not assigned to
>> > someone
>> > > >> as
>> > > >> > > the
>> > > >> > > >>>>>> next
>> > > >> > > >>>>>>>>>>> couple
of weeks pass by I'll begin moving at least
>> > C++ and
>> > > >> > > Python
>> > > >> > > >>>>>>>>>>> issues
to 0.14 that don't seem like they're going to
>> > get
>> > > >> done
>> > > >> > > for
>> > > >> > > >>>>>>>>>>> 0.13. If
development stakeholders for C#, Java, Rust,
>> > Ruby,
>> > > >> > and
>> > > >> > > >>>>>> other
>> > > >> > > >>>>>>>>>>> components
can review and curate the issues that
>> > would be
>> > > >> > > >>>>>> helpful.
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> You can
help keep the JIRA issues tidy by making sure
>> > to
>> > > >> add
>> > > >> > > Fix
>> > > >> > > >>>>>>>>>>> Version
to issues and to make sure to add a Component
>> > so
>> > > >> that
>> > > >> > > >>>>>> issues
>> > > >> > > >>>>>>>>>>> are properly
categorized in the release notes.
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> Thanks
>> > > >> > > >>>>>>>>>>> Wes
>> > > >> > > >>>>>>>>>>>
>> > > >> > > >>>>>>>>>>> On Sat,
Feb 9, 2019 at 10:39 AM Wes McKinney <
>> > > >> > > >>>>>> wesmckinn@gmail.com>
>> > > >> > > >>>>>>>>>> wrote:
>> > > >> > > >>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>> See
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>
>> > > >> > > >>>>>>
>> > > >> > >
>> > > >> >
>> > > >>
>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> > > >> > > >>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>> The
source release step is one of the places where
>> > > >> problems
>> > > >> > > >>>>>> occur.
>> > > >> > > >>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>> On
Sat, Feb 9, 2019, 10:33 AM <aneesh@quiltdata.io
>> > wrote:
>> > > >> > > >>>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>>>>
On Feb 8, 2019, at 9:19 AM, Uwe L. Korn <
>> > > >> mail@uwekorn.com
>> > > >> > >
>> > > >> > > >>>>>>>> wrote:
>> > > >> > > >>>>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>>>>
We could dockerize some of the release steps to
>> > ensure
>> > > >> > that
>> > > >> > > >>>>>> they
>> > > >> > > >>>>>>>>>> run in the
same environment.
>> > > >> > > >>>>>>>>>>>>>
>> > > >> > > >>>>>>>>>>>>>
I may be able to help with said Dockerization. If
>> > not for
>> > > >> > > this
>> > > >> > > >>>>>>>>>> release, then
for the next. Are there docs on which
>> > systems
>> > > >> we
>> > > >> > > >>>>>> wish to
>> > > >> > > >>>>>>>>>> target and/or
any build steps beyond the current dev
>> > > >> > container (
>> > > >> > > >>>>>>>>>>
>> > https://github.com/apache/arrow/tree/master/dev/container)?
>> > > >> > > >>>>>>>>>>
>> > > >> > > >>>>>>>>
>> > > >> > > >>>>>>
>> > > >> > >
>> > > >> >
>> > > >>
>> >
Mime
View raw message