Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <675B514A-C19F-42D1-8A6A-F3240A11B26A@firemirror.com>
References: <CABYbY7c=xDDCJkrK7aXieihm5Z5bNYy8Xq5r-dyAEp7TyxKo5w@mail.gmail.com>
 <CABYbY7c0PRTLzq8+UTFUvCGN31tLT1=EApr66eK6y=LL-nZ8Kw@mail.gmail.com>
 <A50D0410-C61A-4901-81C2-5F3EDF86B312@firemirror.com> <34B5E48F-1484-4B29-99F0-1249395B5D82@gmail.com>
 <CABDm2Q9F5yqEhRS5uJPD2X3W0x3bD6tzGFP=_Fikjykx50=xJQ@mail.gmail.com>
 <ED4CEEC4-2591-479F-A35F-45D02881C964@firemirror.com> <0BB08CF6-FCCA-45B8-91FF-078A5027E8B2@firemirror.com>
 <272D08EE-EA47-4B7C-AEE8-D67811F8908E@firemirror.com> <ED826700-1483-4446-A52E-F58D3434A9F8@modernizingmedicine.com>
 <CAC15uRvaDYk_05EbMCVmvUW2EwNuxHjYFbugVc7axuWFPS272w@mail.gmail.com>
 <99B82D60-359B-4FA2-8985-C6A218B00209@modernizingmedicine.com>
 <CAMNEcwvEwe27XrE-mgi0uHrcsqA_tNszZS-cu15pgdfHmgJ9kA@mail.gmail.com>
 <CAF+3fpO5XdEw8ZvXJxttwd4CX83zCEnjQigBR2X0GwxY2n0-0w@mail.gmail.com>
 <CAF+3fpM+SMrGo9E_rLpy3OrAnmR+u=RPdsvjgOnD1TECPTQx4A@mail.gmail.com>
 <CAF+3fpMB8KoF=GCv4+Ld3OKSWx6yk9naDor-0j+fbAb=1UURSw@mail.gmail.com>
 <135FA26C-4F7D-4353-A7F8-A8F300D337BB@gmail.com> <CABDm2Q887ycS8EfRvS6ri0qA321ZZFHVp0GTRuN4FA9N=LQ5dQ@mail.gmail.com>
 <CAF+3fpNp=OZK0uWwawN1Yz2JoUO7CUmcDS1zaMYvJmcyvn2h5g@mail.gmail.com>
 <CAMNEcwsf7ZTAT8WEL9WKL9yY9Z=GOzULtP6BEHPiB=rxu98NCg@mail.gmail.com> <675B514A-C19F-42D1-8A6A-F3240A11B26A@firemirror.com>
From: Arthur Wiedmer <arthur.wiedmer@gmail.com>
Date: Fri, 10 Nov 2017 09:18:12 -0800
Message-ID: <CAF+3fpMyALOgchikbFi-_fCbnRCuvYW3S1t06uVZiod3js8WmQ@mail.gmail.com>
Subject: Re: [VOTE] Airflow 1.9.0rc1
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a11442970d355c0055da41ce0"
archived-at: Fri, 10 Nov 2017 17:18:49 -0000

--001a11442970d355c0055da41ce0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Ash,

There seem to be some solutions, but there are pretty hacky and poorly
documented :

https://stackoverflow.com/questions/18026980/python-setuptools-how-can-i-li=
st-a-private-repository-under-install-requires
https://github.com/pypa/pip/issues/2124

That said, we should be able to figure out a path :)

Maybe we can get the incubator to reconsider for this particular case.

Best,
Arthur

On Fri, Nov 10, 2017 at 1:25 AM, Ash Berlin-Taylor <
ash_airflowlist@firemirror.com> wrote:

> The other difference is that if you depend upon airflow in a module where
> you want to put it in the install_requires section of a setup.py (not an
> application which has a requirements.txt, say) you can't use a git tag. O=
r
> at least I couldn't get it working.
>
> It doesn't make a difference a lot of time, but it is occasionally useful=
.
>
> -ash
>
> > On 9 Nov 2017, at 23:08, Alek Storm <alek.storm@gmail.com> wrote:
> >
> > It=E2=80=99s not a major difference, but installing from a git repo via=
 pip
> > requires a completely different syntax, which complicates our tooling,
> e.g.:
> >
> > $ pip install 'apache-airflow[postgres,celery,rabbitmq]=3D=3D{{version}=
}'
> >
> > $ pip install 'git+git://github.com/apache/
> incubator-airflow@{{version}}#egg=3Dapache-airflow[postgres,celery,rabbit=
mq]
> '
> >
> > Alek
> > =E2=80=8B
> >
> > On Thu, Nov 9, 2017 at 3:53 PM, Arthur Wiedmer <arthur.wiedmer@gmail.co=
m
> >
> > wrote:
> >
> >> I agree with Bolke that it would be better to provide dev releases in
> PyPI,
> >> but my understanding was that, while not an official release channel, =
it
> >> still has the apache branding and we should be careful nonetheless.
> >>
> >> I am still confused as to why installing from a git tag or the like is
> not
> >> OK for testing, provided our release artifact creation process is
> >> consistent.
> >>
> >> Best,
> >> Arthur
> >>
> >> On Thu, Nov 9, 2017 at 12:09 PM, Daniel Huang <dxhuang@gmail.com>
> wrote:
> >>
> >>> This is how pip handles RC/beta versions:
> >>>
> >>>
> >>>> Pre-release Versions
> >>>> Starting with v1.4, pip will only install stable versions as specifi=
ed
> >> by
> >>>> PEP426 by default. If a version cannot be parsed as a compliant PEP4=
26
> >>>> version then it is assumed to be a pre-release.
> >>>> If a Requirement specifier includes a pre-release or development
> >> version
> >>>> (e.g. >=3D0.0.dev0) then pip will allow pre-release and development
> >>> versions
> >>>> for that requirement. This does not include the !=3D flag.
> >>>> The pip install command also supports a --pre flag that will enable
> >>>> installing pre-releases and development releases.
> >>>
> >>>
> >>> Source:
> >>> https://pip.pypa.io/en/stable/reference/pip_install/#pre-
> >> release-versions
> >>> <https://pip.pypa.io/en/stable/reference/pip_install/#
> >> pre-release-versions
> >>>>
> >>>
> >>> On Thu, Nov 9, 2017 at 11:54 AM, Bolke de Bruin <bdbruin@gmail.com>
> >> wrote:
> >>>
> >>>> I think we should put this up for discussion. PyPi is not an officia=
l
> >>>> apache channel, so in theory we could put anything on PyPI. I also
> >> think
> >>>> (didn=E2=80=99t confirm) pip doesn=E2=80=99t upgrade to RC/beta etc.
> >>>>
> >>>> Any thoughts?
> >>>>
> >>>> Bolke.
> >>>>
> >>>>> On 9 Nov 2017, at 15:53, Arthur Wiedmer <arthur@apache.org> wrote:
> >>>>>
> >>>>> Hi Alek,
> >>>>>
> >>>>> Technically, we cannot release a distribution on PyPI until we have
> >>> voted
> >>>>> on a release. And here usually a release artifact. It is a little
> >>>>> convoluted in the case of Python, but we are getting the hang of it=
.
> >>>>>
> >>>>> That said, installing from a git reference is a possibility too if
> >> you
> >>>> want
> >>>>> the fastest path to install.
> >>>>>
> >>>>> Best,
> >>>>> Arthur
> >>>>>
> >>>>> On Nov 9, 2017 06:34, "Alek Storm" <alek.storm@gmail.com> wrote:
> >>>>>
> >>>>> I think this has been mentioned before, but it would be much easier
> >> for
> >>>> us
> >>>>> (my team) to test RCs if they were published to PyPI. Or is that
> >>> against
> >>>>> Apache guidelines?
> >>>>>
> >>>>> Alek
> >>>>>
> >>>>> On Thu, Nov 9, 2017 at 8:29 AM, Michael Crawford <
> >>>>> michael.crawford@modernizingmedicine.com> wrote:
> >>>>>
> >>>>>> Thanks.  Yes I understand it isn=E2=80=99t released yet.
> >>>>>>
> >>>>>>
> >>>>>>> On Nov 9, 2017, at 9:09 AM, Driesprong, Fokko <fokko@driesprong.f=
rl
> >>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi Michael,
> >>>>>>>
> >>>>>>> You have to install it from the tar.gz:
> >>>>>>>
> >>>>>>> wget
> >>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.
> >>>>>> 9.0rc1/apache-airflow-1.9.0rc1+incubating-bin.tar.gz
> >>>>>>> pip install /tmp/apache-airflow.tar.gz
> >>>>>>>
> >>>>>>> The steps of updating, are in the UPDATING.md:
> >>>>>>> https://github.com/apache/incubator-airflow/blob/master/
> >> UPDATING.md
> >>>>>>>
> >>>>>>> Please note that 1.9 is not released yet, but you are welcome to
> >> try
> >>>> out
> >>>>>>> RC1.
> >>>>>>>
> >>>>>>> Cheers, Fokko
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
>
>

--001a11442970d355c0055da41ce0--