arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Grove <andygrov...@gmail.com>
Subject Re: Aapche process questions
Date Thu, 05 Apr 2018 02:17:37 GMT
Wes,

We talked about nightlies and downstream dependencies and I just want to be
clear I'm understanding what you are suggesting. Apologies if I'm being a
bit slow.

I think we are in agreement that nightlies make sense for now because
things are moving fast and this Rust library is very new.

I can build nightly releases myself, but if I publish those to crates.io,
they should be under a separate name and not using the "arrow" crate that I
established for this project on crates.io

Is that correct?

Thanks,

Andy.

On Wed, Apr 4, 2018 at 6:29 PM, Wes McKinney <wesmckinn@gmail.com> wrote:

> Hi Andy,
>
> You're free to do whatever you like outside of Apache infrastructure -- I
> don't think we can have the apache/arrow Travis publishing artifacts to
> crates.io, though.
>
> If you want to have some scripts in the repo to help with creating
> nightlies, that's totally fine, but this isn't something that the PMC is
> able to do anything about formally. We documented our nightly build tool
> for Python here
>
> https://github.com/apache/arrow/blob/master/python/doc/
> source/development.rst#nightly-builds-of-arrow-cpp-
> parquet-cpp-and-pyarrow-for-linux
>
> - Wes
>
> On Wed, Apr 4, 2018, 7:13 PM Andy Grove <andygrove73@gmail.com> wrote:
>
> > Wes,
> >
> > Thanks for the feedback.
> >
> > A nightly release sounds appealing in these early days. Technically, I
> > assume this is just a case of configuring travis to update the minor
> > version number and run a "cargo publish" on master once per day (if there
> > are changes)?
> >
> > From a process point of view, what is involved in deciding whether to do
> > this or not?
> >
> > Thanks,
> >
> > Andy.
> >
> >
> >
> >
> > On Wed, Apr 4, 2018 at 8:18 AM, Wes McKinney <wesmckinn@gmail.com>
> wrote:
> >
> > > hi Andy,
> > >
> > > > My argument for doing this is that although other Rust developers can
> > > set up a github dependency in their Cargo,toml, this just isn't a
> natural
> > > way to work in Rust so we are making it hard for people to experiment
> > with
> > > Arrow.
> > >
> > > Being penalized by the language for not using a centralized package
> > > repository seems odd to me, but if that's the way Rust is, then so be
> > > it.
> > >
> > > > 1. Is it ok for me to push updated releases to crates.io?
> > >
> > > If crates.io definitely needs to get updated as frequently as you are
> > > saying, we should make separate Rust releases and hold votes like we
> > > have with JavaScript. It sounds like nightlies or downstream
> > > "releases" would be a better solution while the project is new / in
> > > flux, see below:
> > >
> > > > 2. Is releasing from a fork of the code (under a different name)
> > > acceptable?
> > >
> > > Sure, you can make your own "downstream" releases of the project, as
> > > long as they aren't advertised as being releases made by the Apache
> > > project. Some of us have been building nightly Python packages for
> > > development purposes
> > >
> > > - Wes
> > >
> > > On Wed, Apr 4, 2018 at 10:01 AM, Andy Grove <andygrove73@gmail.com>
> > wrote:
> > > > Hi,
> > > >
> > > > I've been creating some frustrations for myself this week because I'm
> > not
> > > > sure how to work efficiently now that the Rust version of Apache
> Arrow
> > is
> > > > in the official Apache repo.
> > > >
> > > > It seems I have two conflicting requirements:
> > > >
> > > > 1. I want Apache Arrow [Rust] to be a community-driven high quality
> > piece
> > > > of software built by many contributors, with a thoughtful code review
> > > > process.
> > > >
> > > > 2. I want to move fast and innovate on my other open source project
> > which
> > > > now depends on Arrow, and I want to help other projects (such as
> > > > https://github.com/sunchao/parquet-rs) integrate with Arrow.
> > > >
> > > > I have two questions that I need guidance on:
> > > >
> > > > 1. Is it ok for me to push updated releases to crates.io?
> > > >
> > > > We currently have a 0.1.0 release of the Rust library in crates.io (
> > > > https://crates.io/crates/arrow) which was made to reserve the Arrow
> > name
> > > > there (with approval from Wes). The github repo has changed quite a
> bit
> > > > since this release but this is the only version that Rust users can
> > > easily
> > > > pull into their projects as a versioned dependency. Understanding
> that
> > > this
> > > > isn't an official Apache release since it hasn't been through a
> release
> > > > process, is it OK to push new versions of this unofficial release as
> > PRs
> > > > are accepted into the repo? I have the *ability* to do this but I
> don't
> > > > know if I have *approval* to do this.
> > > >
> > > > My argument for doing this is that although other Rust developers can
> > set
> > > > up a github dependency in their Cargo,toml, this just isn't a natural
> > way
> > > > to work in Rust so we are making it hard for people to experiment
> with
> > > > Arrow. The code is available in github and anyone could fork the code
> > and
> > > > make their own release to crates.io but I'd prefer them to use the
> one
> > > we
> > > > make.
> > > >
> > > > 2. Is releasing from a fork of the code (under a different name)
> > > acceptable?
> > > >
> > > > We cannot stop people forking the Arrow repo and making their own
> > > releases
> > > > to crates.io under a different name.
> > > >
> > > > I'm wondering if I should just do that for now e.g. release a
> > > > "datafusion-arrow" project from the branch in my fork where I have
> > merged
> > > > all of my PRs. This way I can keep moving fast and if other projects
> > such
> > > > as parquet-rs want to depend on my releases as a temporary measure,
> > they
> > > > can, until the official Arrow crate catches up. This would fix my
> short
> > > > term pains but I don't want to detract from the official project.
> > > >
> > > > I'd appreciate some guidance on the best way forward.
> > > >
> > > > Thanks,
> > > >
> > > > Andy.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message