airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Aiming for an Apache release 1st week of September?
Date Mon, 11 Jul 2016 14:06:21 GMT
Ok. I created a “branch-1.7.2-apache” branch based on airbnb_rb1.7.1_4 to which I added
cherry-picked commits to establish Apache compliancy (I think!). If we need to update this
branch I prefer very much to have commits that are also applied against master.

- Bolke 


> Op 11 jul. 2016, om 15:16 heeft Bolke de Bruin <bdbruin@gmail.com> het volgende
geschreven:
> 
> Ok. I am seeing a “airbnb_rb1.7.1_4” branch, I am assuming this is the branch to
start from (?). 
> 
> I will work on applying the commits that are required to become compliant.
> 
> - Bolke
> 
> 
>> Op 11 jul. 2016, om 05:22 heeft Maxime Beauchemin <maximebeauchemin@gmail.com>
het volgende geschreven:
>> 
>> +1
>> 
>> On Fri, Jul 8, 2016 at 5:16 PM, Dan Davydov <dan.davydov@airbnb.com.invalid>
>> wrote:
>> 
>>> +1 the minimal apache cherrypick release makes sense to me.
>>> 
>>> On Fri, Jul 8, 2016 at 1:14 PM, Chris Riccomini <criccomini@apache.org>
>>> wrote:
>>> 
>>>> Hey Bolke,
>>>> 
>>>> A fast release with the 1.7.1.3 + cherry picks listed above sounds like
>>> the
>>>> way to go. Then, a second release in sept where we just cut from master.
>>>> 
>>>> I'm +1 on this. Lets us get our Apache ducks in a row without worrying
>>>> about stabilizing everything simultaneously.
>>>> 
>>>> Cheers,
>>>> Chris
>>>> 
>>>> On Fri, Jul 8, 2016 at 12:32 AM, Bolke de Bruin <bdbruin@gmail.com>
>>> wrote:
>>>> 
>>>>> This was my assessment as well, thus I agree. My suggestion is to start
>>>>> the process and see if we get questions about this that require us the
>>>>> change our point of view.
>>>>> 
>>>>> If we do an earlier release I would like to aim for July 19, but that
>>>>> might be a bit short notice. If needed I can put myself up as release
>>>>> manager till the 21st. If we do 1.7.1.3 + cherry picks I would say
>>>>> 
>>>>> * Licenses
>>>>> * Notices
>>>>> * Disclaimer
>>>>> * Highcharts -> d3
>>>>> 
>>>>> Makes sense? Anything missing here?
>>>>> 
>>>>> - Bolke
>>>>> 
>>>>>> Op 7 jul. 2016, om 18:04 heeft Chris Riccomini <
>>> criccomini@apache.org>
>>>>> het volgende geschreven:
>>>>>> 
>>>>>>> but it's acceptable to have a soft dependency on an LGPL component,
>>>> such
>>>>>> that a user could deploy the LGPL component separately to enable
>>>>> additional
>>>>>> optional features
>>>>>> 
>>>>>> This is precisely what I believe is going on with Airflow. It's under
>>>> an
>>>>>> airflow[postgres] package (so `pip install airflow` doesn't even
>>>> install
>>>>>> it). We went through a very similar exercise with Samza, where we
>>> had a
>>>>>> dependency on Paramiko (also LGPL [1]), and our (LinkedIn) lawyers
>>>> talked
>>>>>> to Apache, and agreed that it was fine.
>>>>>> 
>>>>>> [1] https://github.com/paramiko/paramiko/blob/master/LICENSE
>>>>>> 
>>>>>> On Wed, Jul 6, 2016 at 10:16 PM, Chris Nauroth <
>>>> cnauroth@hortonworks.com
>>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> Here are more details on Apache release requirements:
>>>>>>> 
>>>>>>> http://www.apache.org/dev/release-publishing.html
>>>>>>> 
>>>>>>> 
>>>>>>> http://www.apache.org/dev/release
>>>>>>> 
>>>>>>> 
>>>>>>> To summarize, it's much more focused on compliance with licensing,
>>>>> signing
>>>>>>> and Apache infrastructure requirements.  That's the kind of scrutiny
>>>>> that
>>>>>>> a release candidate will get from the Incubator PMC rather than
deep
>>>>>>> testing for verification of new features or bug fixes.
>>>>>>> 
>>>>>>> For that reason, I think it makes sense for a podling's first
Apache
>>>>>>> release to focus on nothing but those ASF policy requirements.
 It's
>>>>>>> completely normal for a podling's early release candidates to
have a
>>>> few
>>>>>>> false starts that get voted down, because the policies are complex
>>> the
>>>>>>> first time around.  Some projects have found it helpful to write
a
>>>> "How
>>>>> to
>>>>>>> Release" web page during the first release, so that they have
>>>>> step-by-step
>>>>>>> notes to follow during subsequent releases.  Focusing on "latest
>>>> stable"
>>>>>>> with a few additional patches sounds like a great plan to me,
>>> because
>>>> it
>>>>>>> decouples the challenges of your first ASF release from other
>>> software
>>>>>>> development pressures, such as pressure from a user base to ship
a
>>> new
>>>>>>> feature quickly.
>>>>>>> 
>>>>>>> Regarding the LGPL question, in general, the answer is that we
are
>>>>>>> prohibited from redistributing any LGPL component, but it's
>>> acceptable
>>>>> to
>>>>>>> have a soft dependency on an LGPL component, such that a user
could
>>>>> deploy
>>>>>>> the LGPL component separately to enable additional optional
>>> features.
>>>>>>> More details are here:
>>>>>>> 
>>>>>>> http://www.apache.org/legal/resolved.html#prohibited
>>>>>>> 
>>>>>>> 
>>>>>>> A specific example of this is Apache Hadoop's integration with
LZO
>>>>>>> compression, which uses a GPL license.  Hadoop does not redistribute
>>>> LZO
>>>>>>> or include any code that is tightly coupled to it, but the Hadoop
>>>>> codebase
>>>>>>> does have a notion of a pluggable CompressionCodec, with
>>>> implementations
>>>>>>> of the interface discoverable at runtime.  This setup supports
users
>>>>>>> downloading and installing a separate LZO integration library
onto
>>>>>>> Hadoop's classpath.
>>>>>>> 
>>>>>>> --Chris Nauroth
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 7/6/16, 9:36 PM, "Maxime Beauchemin" <maximebeauchemin@gmail.com
>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> This sounds very reasonable to me, though we may be able
to do an
>>>>> earlier
>>>>>>>> release as a practice run for an Apache release with a snapshot
of
>>>> our
>>>>>>>> production which would consists of the latest release plus
a set of
>>>>> cherry
>>>>>>>> picked PRs.
>>>>>>>> 
>>>>>>>> How does an Apache release differ from a standard release
again?
>>>>>>>> 
>>>>>>>> Max
>>>>>>>> 
>>>>>>>> On Wed, Jul 6, 2016 at 7:59 PM, Chris Riccomini <
>>>> criccomini@apache.org
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> One other thing to note is that I'm planning to run the
RCs in all
>>>> of
>>>>>>>>> our
>>>>>>>>> environments to exercise things. We should make sure
that we're
>>> all
>>>>>>>>> committed (as well as the community) to burning the RCs
in on our
>>>>>>>>> respective environments for a few weeks.
>>>>>>>>> 
>>>>>>>>> On Wed, Jul 6, 2016 at 7:58 PM, Chris Riccomini <
>>>>> criccomini@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hey Bolke,
>>>>>>>>>> 
>>>>>>>>>>> should we aim for a release 1st week of September
>>>>>>>>>> 
>>>>>>>>>> Sounds good to me!
>>>>>>>>>> 
>>>>>>>>>>> I would want to aim earlier, but due to holidays
I guess it
>>> might
>>>> be
>>>>>>>>>> smarter to schedule it a bit after?
>>>>>>>>>> 
>>>>>>>>>> So would I, personally. I'd be OK with starting RCs
now, to be
>>>> frank.
>>>>>>>>> What
>>>>>>>>>> do others think?
>>>>>>>>>> 
>>>>>>>>>>> Should we vote on the above?
>>>>>>>>>> 
>>>>>>>>>> No need, IMO. A discuss like this is enough, assuming
there are
>>> no
>>>>>>>>>> objections.
>>>>>>>>>> 
>>>>>>>>>> Re: psycopg2, I don't think it's an issue, but we
should probably
>>>>>>>>> file a
>>>>>>>>>> LEGAL [1] just to be sure. In any case, we don't
have to be
>>>> compliant
>>>>>>>>> for a
>>>>>>>>>> release in incubator, I believe. We just need to
be moving in
>>> that
>>>>>>>>>> direction.
>>>>>>>>>> 
>>>>>>>>>> Cheers,
>>>>>>>>>> Chris
>>>>>>>>>> 
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/LEGAL
>>>>>>>>>> 
>>>>>>>>>> On Wed, Jul 6, 2016 at 1:24 PM, Bolke de Bruin <
>>> bdbruin@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> 
>>>>>>>>>>> As I don¹t think there are any show stoppers
to have an Apache
>>>>>>>>> release
>>>>>>>>>>> should we aim for a release 1st week of September?
I would want
>>> to
>>>>>>>>> aim
>>>>>>>>>>> earlier, but due to holidays I guess it might
be smarter to
>>>> schedule
>>>>>>>>> it
>>>>>>>>> a
>>>>>>>>>>> bit after?
>>>>>>>>>>> 
>>>>>>>>>>> - RC last week of August giving about two weeks
to have it run
>>> in
>>>>>>>>>>> production in our environments
>>>>>>>>>>> - Guess voting needs to happen at the IPMC
>>>>>>>>>>> - Release (Champagne!)
>>>>>>>>>>> 
>>>>>>>>>>> Earlier there was some discussion about psycopg2
/ postgres
>>>>>>>>>>> interoperability (psycopg2 being LGPL). Personally
I don¹t think
>>>>>>>>> there
>>>>>>>>> is
>>>>>>>>>>> an issue as we are not redistributing psycopg2
ourselves and we
>>>> are
>>>>>>>>> not
>>>>>>>>>>> dependent on it to function. An company can choose
thus what
>>> they
>>>>>>>>> want
>>>>>>>>>>> without being `tainted` by the LGPL. Any remarks
on this?
>>>>>>>>>>> 
>>>>>>>>>>> Should we vote on the above?
>>>>>>>>>>> 
>>>>>>>>>>> - Bolke
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
> 


Mime
View raw message