airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@apache.org>
Subject Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2
Date Mon, 17 Jul 2017 22:36:28 GMT
The main artifact to vote upon on is a source release. This implies just
creating a clean tarball of the source codebase with an appropriate README,
INSTALL, LICENSE, DISCLAIMER, NOTICE (all of these should be checked in to
the codebase as the release should map to a specific commit hash which can
be re-used to create the source tarball again if needed). License and
Notice files should cover all files in the codebase.

You can also publish binary build(s) as a convenience ( assuming you want
to publish to pypi) - this would be a separate tarball and likely different
notice/license files (would suggest having these in the src repo too) as
the binary tarball will likely have different files.

thanks
-- Hitesh



On Mon, Jul 17, 2017 at 2:18 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Great!
>
> It is also a bit new to me so maybe @Hitesh @Jakob can help with some
> guidance here?
>
> But my assumption indeed is:
>
> 1. Make a tarball from the repo with build instructions (including a
> working License check!) -> Vote here and IPMC. This is the “official”
> release.
> 2. Make sdist for redistribution on PyPi
>
> Bdist isn’t required.
>
> Cheers
> Bolke.
>
> > On 17 Jul 2017, at 06:27, Maxime Beauchemin <maximebeauchemin@gmail.com>
> wrote:
> >
> > I've been slammed but skies are clearing up now I'm hoping.
> >
> > Reading the general@ thread I'm unclear about the next steps, targz the
> > whole repo and add build instructions? What should the file with the
> build
> > instructions be called? How to label that new tarbar? Can we skip the
> bdist?
> >
> > Max
> >
> > On Sun, Jul 16, 2017 at 12:35 PM, Bolke de Bruin <bdbruin@gmail.com
> <mailto:bdbruin@gmail.com>> wrote:
> >
> >> Max, Ping? Do you need help?
> >>
> >>> On 9 Jul 2017, at 14:30, Bolke de Bruin <bdbruin@gmail.com> wrote:
> >>>
> >>> Hi Max,
> >>>
> >>> The canonical distribution would be what we have in git right now (ie.
> >> before running python sdist). The rest is just convenience packages. So
> >> npm, would solve the issue as long as we don’t rely on any non APL
> >> compatible packages in core. I don’t think npm/yarn/webpack needs to be
> >> done for 1.8.3, but considering the messy javascript that we currently
> have
> >> it would be nice to put it on the todo.
> >>>
> >>> Cheers
> >>> Bolke
> >>>
> >>>> On 9 Jul 2017, at 06:46, Maxime Beauchemin <
> maximebeauchemin@gmail.com
> >> <mailto:maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>>>
> wrote:
> >>>>
> >>>> As far as I understand npm would not solve the problem as typically
> we'd
> >>>> build our "entry" files and distribute that with Airflow as static
> >> files.
> >>>> Those entry files would contain these other npm packages, minified.
> >> (from
> >>>> my understanding that is the same issue as packaging the libs
> >> themselves)
> >>>>
> >>>> To make them runtime deps would be atypical and more complicated.
> >> `airflow
> >>>> webserver` would need to "build" (npm install/webpack) and the
> webserver
> >>>> would have to serve these static files out of some that temp location
> >>>> (perhaps ~/.airflow/airflow.entry.js) as opposed to out of
> >> `site-packages`.
> >>>>
> >>>> Also note that Airflow's javascript is in pretty bad shape (scattered
> in
> >>>> jinja templates files) and it would take quite a significant amount
of
> >> work
> >>>> to move to using npm/webpack.
> >>>>
> >>>> I'm back from vacation and will have things to catch up on next week
> but
> >>>> I'll try to find time to look into some of this.
> >>>>
> >>>> On Thu, Jul 6, 2017 at 1:10 PM, Bolke de Bruin <bdbruin@gmail.com
> <mailto:bdbruin@gmail.com>
> >> <mailto:bdbruin@gmail.com <mailto:bdbruin@gmail.com>>> wrote:
> >>>>
> >>>>> Hi Folks,
> >>>>>
> >>>>> We probably need to adjust our release process as can be observed
in
> >> the
> >>>>> IPMC thread. As we are packaging a “sdist” it does not pass
license
> >> checks
> >>>>> and one cannot verify the validity of what we are doing. It was
> >> suggested
> >>>>> by one of the maintainers of another python project to create 3
> >> different
> >>>>> packages:
> >>>>>
> >>>>> 1. A source tarball which is essentially a snapshot of the repository
> >>>>> 2. A sdist
> >>>>> 3. A bdist
> >>>>>
> >>>>> 1 should then be the canonical Apache release. It should be
> >> accompanied by
> >>>>> build instructions and it should pass RAT checks. This is the package
> >> we
> >>>>> will vote on.
> >>>>> 2 is what we have voted upon until now. It should contain (it does)
> >>>>> LICENSE, NOTICE, and DISCLAIMER
> >>>>> 3 bdist, wheel package. Same as 2. Not really required, but more
a
> >>>>> convenience package as is 2
> >>>>>
> >>>>> 2 and 3 can be published to PyPi.
> >>>>>
> >>>>> Max: can you take care of this? We need to vote on 1.  Build
> >> instruction
> >>>>> could be added to an INSTALL file or just to the README.md file?
See
> >> for
> >>>>> inspiration the GitHub page of ariatosca: https://github.com/apache/
> <https://github.com/apache/> <
> >> https://github.com/apache/ <https://github.com/apache/>>
> >>>>> incubator-ariatosca <https://github.com/apache/incubator-ariatosca
<
> https://github.com/apache/incubator-ariatosca> <
> >> https://github.com/apache/incubator-ariatosca <
> https://github.com/apache/incubator-ariatosca>>>
> >>>>>
> >>>>> In the meantime I am figuring out an issue with one of the
> >> dependencies of
> >>>>> nvd3 which might be or have been GPL3 which is incompatible with
the
> >> APL,
> >>>>> which we are distributing together with our source. Ideally, we
> should
> >> move
> >>>>> to a “yarn/webpack” build which will resolve those issues
> >> automatically as
> >>>>> these become runtime dependencies then in case of 1,2,3.
> >>>>>
> >>>>> Cheers
> >>>>> Bolke
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 6 Jul 2017, at 05:20, Sumit Maheshwari <sumeet.manit@gmail.com
> <mailto:sumeet.manit@gmail.com>
> >> <mailto:sumeet.manit@gmail.com <mailto:sumeet.manit@gmail.com>>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Awesome.. thanks a lot Max for taking the RM responsibility..
> >>>>>>
> >>>>>> On Jul 5, 2017 11:10 PM, "Chris Riccomini" <criccomini@apache.org
> <mailto:criccomini@apache.org>
> >> <mailto:criccomini@apache.org <mailto:criccomini@apache.org>>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Nice, thanks Max!
> >>>>>>>
> >>>>>>> On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin <
> >>>>>>> maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>
> <mailto:maximebeauchemin@gmail.com <mailto:maximebeauchemin@gmail.com>>>
> >> wrote:
> >>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.
> >>>>>>>>
> >>>>>>>> 5 “+1” votes received, 4 out of which are "binding":
> >>>>>>>>
> >>>>>>>> - Maxime Beauchemin (binding)
> >>>>>>>> - Bolke de Bruin (binding)
> >>>>>>>> - Chris Riccomini (binding)
> >>>>>>>> - Sumit Maheshwari  (binding)
> >>>>>>>>
> >>>>>>>> - Kengo Seki  (non-binding)
> >>>>>>>>
> >>>>>>>> Vote thread (start):
> >>>>>>>> *http://mail-archives.apache.org/mod_mbox/incubator-
<
> http://mail-archives.apache.org/mod_mbox/incubator-> <
> >> http://mail-archives.apache.org/mod_mbox/incubator- <
> http://mail-archives.apache.org/mod_mbox/incubator->>
> >>>>>>>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> >>>>>>>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com <http://40mail.gmail.com/>
> <http://40mail.gmail.com/ <http://40mail.gmail.com/>
> >>> %3E
> >>>>>>>> <http://mail-archives.apache.org/mod_mbox/incubator-
<
> http://mail-archives.apache.org/mod_mbox/incubator-> <
> >> http://mail-archives.apache.org/mod_mbox/incubator- <
> http://mail-archives.apache.org/mod_mbox/incubator->>
> >>>>>>>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> >>>>>>>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com <http://40mail.gmail.com/>
> <http://40mail.gmail.com/ <http://40mail.gmail.com/>
> >>> %3E>*
> >>>>>>>>
> >>>>>>>> Next steps:
> >>>>>>>> 1) will start the voting process at the IPMC mailinglist.
I don’t
> >>>>> expect
> >>>>>>>> changes.
> >>>>>>>> 2) Only after the positive voting on the IPMC and finalisation
I
> >> will
> >>>>>>>> rebrand the RC to Release.
> >>>>>>>> 3) I will upload it to the incubator release page, then
the tar
> ball
> >>>>>>> needs
> >>>>>>>> to propagate to the mirrors.
> >>>>>>>> 4) Update the website
> >>>>>>>> 5) Finally, will upload to Pypi.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>>
> >>>>>>>> Max
> >>>>>>>>
> >>>>>>>> On Mon, Jul 3, 2017 at 12:11 PM, Bolke de Bruin <
> bdbruin@gmail.com <mailto:bdbruin@gmail.com>
> >> <mailto:bdbruin@gmail.com <mailto:bdbruin@gmail.com>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Ping?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On 30 Jun 2017, at 14:21, Bolke de Bruin <bdbruin@gmail.com
> <mailto:bdbruin@gmail.com>
> >> <mailto:bdbruin@gmail.com <mailto:bdbruin@gmail.com>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Max,
> >>>>>>>>>>
> >>>>>>>>>> I think you can close the vote?
> >>>>>>>>>>
> >>>>>>>>>> Bolke
> >>>>>>>>>>
> >>>>>>>>>>> On 27 Jun 2017, at 02:45, Kengo Seki <sekikn@apache.org
> <mailto:sekikn@apache.org> <mailto:
> >> sekikn@apache.org <mailto:sekikn@apache.org>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> +1 (non-binding)
> >>>>>>>>>>>
> >>>>>>>>>>> - verified signatures and checksums
> >>>>>>>>>>> - ran scheduler and webserver, confirmed
they worked fine
> >>>>>>>>>>> - confirmed the latest fix on v1.8 branch
(AIRFLOW-809) is
> >> included
> >>>>>>>>>>>
> >>>>>>>>>>> Kengo Seki <sekikn@apache.org <mailto:sekikn@apache.org>
> <mailto:sekikn@apache.org <mailto:sekikn@apache.org>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-06-27 8:53 GMT+09:00 Chris Riccomini
<
> criccomini@apache.org <mailto:criccomini@apache.org>
> >> <mailto:criccomini@apache.org <mailto:criccomini@apache.org>>>:
> >>>>>>>>>>>> +1 (binding)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Been running in our dev env, and everything
looks good.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jun 26, 2017 at 3:00 PM, Alex
Guziel <
> >>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
<mailto:
> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>>
> >>>>>>>> .
> >>>>>>>>> invalid
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Yeah that makes sense. It pages
by default at 500 so it
> >> explains
> >>>>>>> why
> >>>>>>>>> we saw
> >>>>>>>>>>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 26, 2017 at 2:53 PM,
Chris Riccomini <
> >>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
<mailto:
> criccomini@apache.org <mailto:criccomini@apache.org>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> In 1.8.1, the "DAGs" page has
"Show <blank> entries". In
> >> 1.8.2,
> >>>>>>> it
> >>>>>>>>> has
> >>>>>>>>>>>>>> "Show <25> entries". So
it looks like prior to 1.8.2, the
> >>>>>>>> pagination
> >>>>>>>>> was
> >>>>>>>>>>>>>> broken in the sense that it
defaulted to the whole list. We
> >> have
> >>>>>>>> 479
> >>>>>>>>> DAGs
> >>>>>>>>>>>>>> in one env, and it shows them
all. It looks like someone
> fixed
> >>>>>>> the
> >>>>>>>>> entry
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>> default to 25 now, which exposed
the problem for our
> >>>>>>> environments.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 26, 2017 at 2:47
PM, Alex Guziel <
> >>>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
<mailto:
> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>>
> >>>>>>>>> .
> >>>>>>>>>>>>>> invalid
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We're running 1.8.0 + some
extras, and none of us added
> >>>>>>> pagination
> >>>>>>>>>>>>>>> recently, and our homepage
is paginated. Are you sure it's
> >> not
> >>>>>>> the
> >>>>>>>>>>>>> number
> >>>>>>>>>>>>>>> of dags crossing the threshold?
Maybe it's some Flask
> version
> >>>>>>>> thing?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 26, 2017 at
2:45 PM, Chris Riccomini <
> >>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> <mailto:criccomini@apache.org <mailto:criccomini@apache.org>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, I did the 1.8.1
release.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Jun 26, 2017
at 2:44 PM, Alex Guziel <
> >>>>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
<mailto:
> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>>
> >>>>>>>>>>>>> .
> >>>>>>>>>>>>>>>> invalid
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> There's no pagination
in 1.8.1? Are you sure?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 26,
2017 at 2:37 PM, Chris Riccomini <
> >>>>>>>>>>>>>>> criccomini@apache.org <mailto:criccomini@apache.org>
> <mailto:criccomini@apache.org <mailto:criccomini@apache.org>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> It's not happening
on 1.8.1 (since there's no pagination
> >> in
> >>>>>>>> that
> >>>>>>>>>>>>>>>>> version),
> >>>>>>>>>>>>>>>>>> so I'd count
this as a regression. I wouldn't say it's
> >>>>>>>> blocking,
> >>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> pretty ugly.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Jun
26, 2017 at 2:34 PM, Alex Guziel <
> >>>>>>>>>>>>>> alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>
> <mailto:alex.guziel@airbnb.com <mailto:alex.guziel@airbnb.com>>
> >>>>>>>>>>>>>>> .
> >>>>>>>>>>>>>>>>>> invalid
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I'm not
so sure this is a new issue. I think we've seen
> >> it
> >>>>>>> on
> >>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>> production
for quite a while.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon,
Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> >>>>>>>>>>>>>>>>> criccomini@apache.org
<mailto:criccomini@apache.org>
> <mailto:criccomini@apache.org <mailto:criccomini@apache.org>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I am
seeing a strange UI behavior on 1.8.2.RC2. I've
> >>>>>>> opened a
> >>>>>>>>>>>>>>> JIRA
> >>>>>>>>>>>>>>>>>> here:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1348
<
> https://issues.apache.org/jira/browse/AIRFLOW-1348> <
> >> https://issues.apache.org/jira/browse/AIRFLOW-1348 <
> https://issues.apache.org/jira/browse/AIRFLOW-1348>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Has
anyone else seen this?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon,
Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> >>>>>>>>>>>>>>>>>>> sumeet.manit@gmail.com
<mailto:sumeet.manit@gmail.com>
> <mailto:sumeet.manit@gmail.com <mailto:sumeet.manit@gmail.com>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
+1, binding.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> >>>>>>>>>>>>>>>> bdbruin@gmail.com <mailto:bdbruin@gmail.com>
<mailto:
> bdbruin@gmail.com <mailto:bdbruin@gmail.com>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
We have been running it for the last couple of days.
> >> No
> >>>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>
more responsive.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
+1, binding
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
Bolke
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> >>>>>>>>>>>>>>>>>>>>>
maximebeauchemin@gmail.com <mailto:maximebeauchemin@
> gmail.com> <mailto:maximebeauchemin@
> >> gmail.com <http://gmail.com/>>>
> >>>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
Dear all,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
1.8.2 RC2 is baked and available at:
> >>>>>>>>>>>>>>>>>>>>>>>
https://dist.apache.org/repos/ <
> https://dist.apache.org/repos/> <
> >> https://dist.apache.org/repos/ <https://dist.apache.org/repos/>>
> >>>>>>>>>>>>> dist/dev/incubator/airflow
> >>>>>>>>>>>>>> ,
> >>>>>>>>>>>>>>>>> public
> >>>>>>>>>>>>>>>>>>>> keys
> >>>>>>>>>>>>>>>>>>>>>>>
are available
> >>>>>>>>>>>>>>>>>>>>>>>
at https://dist.apache.org/repos/ <
> https://dist.apache.org/repos/> <
> >> https://dist.apache.org/repos/ <https://dist.apache.org/repos/>>
> >>>>>>>>>>>>>>>>> dist/release/incubator/airflow.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
Note that RC1 was the first RC (skipped RC0) and
> was
> >>>>>>>>>>>>>> never
> >>>>>>>>>>>>>>>>>>> announced
> >>>>>>>>>>>>>>>>>>>>>>
since
> >>>>>>>>>>>>>>>>>>>>>>>
it had issues coming out of the oven, so RC2 is the
> >>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>> public
> >>>>>>>>>>>>>>>>>>> RC.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
1.8.2 RC2 is build on to of 1.8.1 with these listed
> >>>>>>>>>>>>>>>> "cherries"
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> top.
> >>>>>>>>>>>>>>>>>>>>>
I
> >>>>>>>>>>>>>>>>>>>>>>>
added the JIRAs that were identified blockers and
> >>>>>>>>>>>>>> targeted
> >>>>>>>>>>>>>>>>>> 1.8.2. I
> >>>>>>>>>>>>>>>>>>>>>>>
attempted to bring in all of the JIRAs that
> targeted
> >>>>>>>>>>>>>> 1.8.2
> >>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>> bailed
> >>>>>>>>>>>>>>>>>>>>>
on
> >>>>>>>>>>>>>>>>>>>>>>>
the ones that were generating merge conflicts. I
> also
> >>>>>>>>>>>>>> added
> >>>>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>
JIRAs that we've been running in production at
> >> Airbnb.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
Issues fixed:
> >>>>>>>>>>>>>>>>>>>>>>>
9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__
> >>>>>>>>>>>>>> ColumnOperator
> >>>>>>>>>>>>>>>> When
> >>>>>>>>>>>>>>>>>>>> Testing
> >>>>>>>>>>>>>>>>>>>>>>>
Booleans
> >>>>>>>>>>>>>>>>>>>>>>>
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all
> >>>>>>>>>>>>>> downstream
> >>>>>>>>>>>>>>>>> tasks
> >>>>>>>>>>>>>>>>>>>>>>>
93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop
> >>>>>>>>>>>>>>> components
> >>>>>>>>>>>>>>>>>>>>>>>
33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a
> >> specific
> >>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>
create
> >>>>>>>>>>>>>>>>>>>>>>>
writable warehouse dir
> >>>>>>>>>>>>>>>>>>>>>>>
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> >>>>>>>>>>>>>>>>>>>>>>>
570b2ed [AIRFLOW-1294] Backfills can loose tasks to
> >>>>>>>>>>>>>> execute
> >>>>>>>>>>>>>>>>>>>>>>>
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE
> >> files
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> match
> >>>>>>>>>>>>>>>>>> ASF
> >>>>>>>>>>>>>>>>>>>>>>>
requirements
> >>>>>>>>>>>>>>>>>>>>>>>
69bd269 [AIRFLOW-1160] Update Spark parameters for
> >>>>>>>>>>>>> Mesos
> >>>>>>>>>>>>>>>>>>>>>>>
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for
> custom
> >>>>>>>>>>>>>>> filters
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> Jinja2
> >>>>>>>>>>>>>>>>>>>>>>>
templates
> >>>>>>>>>>>>>>>>>>>>>>>
6de5330 [AIRFLOW-1119] Fix unload query so headers
> >> are
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>>>>>> row[]
> >>>>>>>>>>>>>>>>>>>>>>>
b4e9eb8 [AIRFLOW-1089] Add Spark application
> >> arguments
> >>>>>>>>>>>>>>>>>>>>>>>
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for
> >> old
> >>>>>>>>>>>>>>> flask
> >>>>>>>>>>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>>>>>>>>>
7a02841 [AIRFLOW-1074] Don't count queued tasks for
> >>>>>>>>>>>>>>>> concurrency
> >>>>>>>>>>>>>>>>>>>> limits
> >>>>>>>>>>>>>>>>>>>>>>>
a2c18a5 [AIRFLOW-1064] Change default sort to
> job_id
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>
TaskInstanceModelView
> >>>>>>>>>>>>>>>>>>>>>>>
d1c64ab [AIRFLOW-1038] Specify celery serialization
> >>>>>>>>>>>>>> options
> >>>>>>>>>>>>>>>>>>>> explicitly
> >>>>>>>>>>>>>>>>>>>>>>>
b4ee88a [AIRFLOW-1036] Randomize exponential
> backoff
> >>>>>>>>>>>>>>>>>>>>>>>
9fca409 [AIRFLOW-993] Update date inference logic
> >>>>>>>>>>>>>>>>>>>>>>>
272c2f5 [AIRFLOW-1167] Support microseconds in
> >> FTPHook
> >>>>>>>>>>>>>>>>>> modification
> >>>>>>>>>>>>>>>>>>>>>
time
> >>>>>>>>>>>>>>>>>>>>>>>
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking
> >> Google
> >>>>>>>>>>>>>>>> BigQuery
> >>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>
acd0166 [AIRFLOW-1263] Dynamic height for charts
> >>>>>>>>>>>>>>>>>>>>>>>
7f33f6e [AIRFLOW-1266] Increase width of gantt y
> axis
> >>>>>>>>>>>>>>>>>>>>>>>
fc33c04 [AIRFLOW-1290] set docs author to 'Apache
> >>>>>>>>>>>>>> Airflow'
> >>>>>>>>>>>>>>>>>>>>>>>
2e9eee3 [AIRFLOW-1282] Fix known event column
> sorting
> >>>>>>>>>>>>>>>>>>>>>>>
2389a8a [AIRFLOW-1166] Speed up
> >>>>>>>>>>>>>>>> _change_state_for_tis_without_
> >>>>>>>>>>>>>>>>>>> dagrun
> >>>>>>>>>>>>>>>>>>>>>>>
bf966e6 [AIRFLOW-1192] Some enhancements to
> >>>>>>>>>>>>>> qubole_operator
> >>>>>>>>>>>>>>>>>>>>>>>
57d5bcd [AIRFLOW-1281] Sort variables by key field
> by
> >>>>>>>>>>>>>>> default
> >>>>>>>>>>>>>>>>>>>>>>>
802fc15 [AIRFLOW-1244] Forbid creation of a pool
> with
> >>>>>>>>>>>>>> empty
> >>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>>>>>
1232b6a [AIRFLOW-1243] DAGs table has no default
> >>>>>>>>>>>>> entries
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> show
> >>>>>>>>>>>>>>>>>>>>>>>
b0ba3c9 [AIRFLOW-1227] Remove empty column on the
> >> Logs
> >>>>>>>>>>>>>> view
> >>>>>>>>>>>>>>>>>>>>>>>
c406652 [AIRFLOW-1226] Remove empty column on the
> >> Jobs
> >>>>>>>>>>>>>> view
> >>>>>>>>>>>>>>>>>>>>>>>
51a83cc [AIRFLOW-1199] Fix create modal
> >>>>>>>>>>>>>>>>>>>>>>>
cac7d4c [AIRFLOW-1200] Forbid creation of a
> variable
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>> empty
> >>>>>>>>>>>>>>>>>>>> key
> >>>>>>>>>>>>>>>>>>>>>>>
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances
> by
> >>>>>>>>>>>>>>>>>>> execution_date
> >>>>>>>>>>>>>>>>>>>>>>>
f446c08 [AIRFLOW-1145] Fix closest_date_partition
> >>>>>>>>>>>>>> function
> >>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>
set
> >>>>>>>>>>>>>>>>>>>>>>>
to True If we're looking for the closest date
> before,
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>> take
> >>>>>>>>>>>>>>>>>>>>>
the
> >>>>>>>>>>>>>>>>>>>>>>>
latest date in the list of date before.
> >>>>>>>>>>>>>>>>>>>>>>>
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for
> >>>>>>>>>>>>>>>>>>> test_csrf_rejection
> >>>>>>>>>>>>>>>>>>>>>>>
bb56805 [AIRFLOW-1170] DbApiHook insert_rows
> inserts
> >>>>>>>>>>>>>>>> parameters
> >>>>>>>>>>>>>>>>>>>>>>
separately
> >>>>>>>>>>>>>>>>>>>>>>>
093b2f0 [AIRFLOW-1150] Fix scripts execution in
> >>>>>>>>>>>>> sparksql
> >>>>>>>>>>>>>>>> hook[]
> >>>>>>>>>>>>>>>>>>>>>>>
777f181 [AIRFLOW-1168] Add closing() to all
> >> connections
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> cursors
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
Max
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message