From dev-return-8915-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Thu Jul 18 16:31:22 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id CD78718063F for ; Thu, 18 Jul 2019 18:31:21 +0200 (CEST) Received: (qmail 85678 invoked by uid 500); 18 Jul 2019 16:31:20 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 85667 invoked by uid 99); 18 Jul 2019 16:31:20 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jul 2019 16:31:20 +0000 Received: from themisto-2.localdomain (231.25.169.217.in-addr.arpa [217.169.25.231]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 551C08234 for ; Thu, 18 Jul 2019 16:31:19 +0000 (UTC) From: Ash Berlin-Taylor Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: [VOTE] Release Airflow 1.10.4 from RC3 Date: Thu, 18 Jul 2019 17:31:16 +0100 References: <8FF14302-46AD-4B30-9D5D-D88C34FA5667@apache.org> <339E39CB-124F-4617-87C0-674D592C51B2@apache.org> <04FCA30F-BBB9-4FD4-A288-C75FE4462CC0@gmail.com> To: dev@airflow.apache.org In-Reply-To: Message-Id: <91AE5BD0-F79F-4EED-9F87-0BDE61A5CB0F@apache.org> X-Mailer: Apple Mail (2.3445.104.11) Errk, that's not good. When you say orphaned do you mean zombie processes owned by pid 1, or = just "process that are hanging around?" https://github.com/apache/airflow/pull/5605 might help if it's the later = (but I suspect it isn't) We've also had reports of the scheduler having a memory leak since = ~1.10.2 https://issues.apache.org/jira/browse/AIRFLOW-4593 so that may = be part of the OOMing too. (If this is the cause of your problem then .4 = isn't worse than .3 I don't think?) -ash > On 18 Jul 2019, at 15:54, James Meickle = wrote: >=20 > Hi folks, >=20 > Sorry to throw a wrench into this, but we're found that this release = is > leaving orphaned processes that eventually OOMing the scheduler. 4 = clusters > have this problem post-upgrade, while 1 non-upgraded cluster doesn't = have > this problem. >=20 > In our case we're using Supervisor to manage scheduler restarts. I am = still > investigating whether this is an interaction with our unique = Supervisor > config, or if it's actually purely an Airflow problem. This switch to > multiprocessing.Manager looks like a promising avenue of investigation > though: > = https://github.com/apache/airflow/compare/1.10.3...1.10.4rc3#diff-ced4fd65= ce02db58eed692eef6e01d05L495 >=20 >=20 >=20 >=20 >=20 > On Wed, Jul 17, 2019 at 3:35 PM Bolke de Bruin = wrote: >=20 >> +1, binding. >>=20 >> Thanks ash! >>=20 >> Verstuurd vanaf mijn iPad >>=20 >>> Op 17 jul. 2019 om 20:45 heeft Kaxil Naik het >> volgende geschreven: >>>=20 >>> +1 (binding) Ran DAGs in both UIs. LGTM >>>=20 >>> Great Job Ash. Appreciate all the effort you put into this. >>>=20 >>> Regards, >>> Kaxil >>>=20 >>> On Wed, Jul 17, 2019 at 10:30 PM Felix Uellendall = >>=20 >>> wrote: >>>=20 >>>> +1 (non-binding) flawlessly passed all my tests via rbac and = classic ui. >>>> Also tested it on production-level dags. Great job Ash, thanks :) >>>>=20 >>>> Kind regards, >>>> Felix >>>>=20 >>>> Sent from ProtonMail mobile >>>>=20 >>>> -------- Original Message -------- >>>>> On Jul 17, 2019, 16:41, Ash Berlin-Taylor wrote: >>>>>=20 >>>>> Thanks Andrii - I missed it to when reviewing the PR so it's my = fault >>>> too. Make the PR against the v[1-10](tel:110)-stable branch please. >>>>>=20 >>>>> -ash >>>>>=20 >>>>>> On 17 Jul [2019](tel:2019), at 14:25, Andrii Soldatenko < >>>> andrii.soldatenko@gmail.com> wrote: >>>>>>=20 >>>>>> @Ash, i'll fix wrong section bug. >>>>>>=20 >>>>>> Sorry about that. >>>>>>=20 >>>>>> On Wed, Jul 17, [2019](tel:2019) at 4:11 PM Robin Edwards < >>>> rob@bidnamic.com> wrote: >>>>>>=20 >>>>>>> +1 (none binding) - been running in production since RC2. >>>>>>>=20 >>>>>>> Thanks for all your hard work >>>>>>>=20 >>>>>>> R >>>>>>>=20 >>>>>>> On Tue, 16 Jul [2019](tel:2019) at 21:15, Ash Berlin-Taylor < >>>> ash@apache.org> wrote: >>>>>>>>=20 >>>>>>>> Thanks for testing. >>>>>>>>=20 >>>>>>>> On 1) everyone should run upgradedb on every upgrade. The = behaviour >> of >>>>>>> not running it wasn't great. >>>>>>>>=20 >>>>>>>> 2) I thought we set deprecation on the ES logging config vars, >> except >>>> we >>>>>>> put the depreciation under the wrong section: >>>>>>>=20 >>>>=20 >> = https://github.com/apache/airflow/blob/1.10.4rc3/airflow/configuration.py#= L147-L153 >>>>>>> :( >>>>>>>>=20 >>>>>>>> On 16 July [2019 21](tel:201921):06:40 BST, James Meickle < >>>> jmeickle@quantopian.com.INVALID> >>>>>>> wrote: >>>>>>>>> +1 (nonbinding) to the release, it fixes a lot of UI issues = we've >>>> been >>>>>>>>> seeing lately. Though two notes: >>>>>>>>>=20 >>>>>>>>> 1) Tasks were unscheduleable until I ran an upgradedb due the >> default >>>>>>>>> pool >>>>>>>>> change. >>>>>>>>>=20 >>>>>>>>> 2) I got crash loops because I based our custom logging file = off of >>>> the >>>>>>>>> previous version's template. The change to the Elasticsearch = conf >>>>>>>>> variables >>>>>>>>> broke this, even though we don't directly use Elasticsearch, = since >> it >>>>>>>>> was >>>>>>>>> now looking for undefined conf variables. It was confusing = until I >>>>>>>>> hunted >>>>>>>>> around the diff and backported the changes in the default = template >> to >>>>>>>>> the >>>>>>>>> template that lives in our repo. >>>>>>>>>=20 >>>>>>>>> On Mon, Jul 15, [2019](tel:2019) at 10:17 AM Ash Berlin-Taylor = < >>>> ash@apache.org> >>>>>>>>> wrote: >>>>>>>>>=20 >>>>>>>>>> Hello Airflow community, >>>>>>>>>>=20 >>>>>>>>>> This email is calling a vote on the release, which will last = for >> 72 >>>>>>>>> hours >>>>>>>>>> (2019-07-08 15:15 Z), and until three binding votes have been >> cast. >>>>>>>>>> Consider this my (binding) +1. >>>>>>>>>>=20 >>>>>>>>>> Airflow [1.10.4](tel:1104) RC3 is available at: >>>>>>>>>> https://dist.apache.org/repos/dist/dev/airflow/1.10.4rc3/ >>>>>>>>>>=20 >>>>>>>>>> *apache-airflow-[1.10.4](tel:1104)rc3-source.tar.gz* is a = source >>>> release that >>>>>>>>> comes >>>>>>>>>> with INSTALL instructions. >>>>>>>>>> *apache-airflow-[1.10.4](tel:1104)rc3-bin.tar.gz* is the = binary >>>> Python "sdist" >>>>>>>>> release. >>>>>>>>>> *apache_airflow-[1.10.4](tel:1104)rc3-py2.py3-none-any.whl* = is the >>>> binary Python >>>>>>>>>> "wheel" release. >>>>>>>>>>=20 >>>>>>>>>> For connivence of testers the RC is on PYPI too. It can be >> installed >>>>>>>>> with: >>>>>>>>>>=20 >>>>>>>>>> pip install 'apache-airflow=3D=3D[1.10.4](tel:1104)rc3' >>>>>>>>>>=20 >>>>>>>>>> Public keys are available at: >>>>>>>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS >>>>>>>>>>=20 >>>>>>>>>> Only votes from PMC members are binding (sorry committers), = but >>>>>>>>> members of >>>>>>>>>> the community are encouraged to test the release and vote = with >>>>>>>>>> "(non-binding)". >>>>>>>>>>=20 >>>>>>>>>> Please note that the version number excludes the `rcX` = string, so >>>>>>>>> it's now >>>>>>>>>> simply [1.10.4](tel:1104). This will allow us to rename the >>>> artefact without >>>>>>>>> modifying >>>>>>>>>> the checksums when we actually release. >>>>>>>>>>=20 >>>>>>>>>> [ ] +1 Release this package as Apache Airflow = [1.10.4](tel:1104) >>>>>>>>>> [ ] 0 No opinion >>>>>>>>>> [ ] -1 Do not release this package because... >>>>>>>>>>=20 >>>>>>>>>> Changes since RC2: >>>>>>>>>>=20 >>>>>>>>>> [AIRFLOW-XXX] Update changelog for [1.10.4](tel:1104)rc3 [Ash >>>> Berlin-Taylor] >>>>>>>>>> [AIRFLOW-XXX] Disable intersphinx loading of `requests` = modules >>>>>>>>> (#[5590](tel:5590)) >>>>>>>>>> [Ash Berlin-Taylor] >>>>>>>>>> [AIRFLOW-[4797](tel:4797)] Improve performance and behaviour = of >>>> zombie detection >>>>>>>>>> (#[5511](tel:5511)) [Stefan Seelmann] >>>>>>>>>> [AIRFLOW-[4911](tel:4911)] Silence the FORBIDDEN errors from = the >>>>>>>>> KubernetesExecutor >>>>>>>>>> (#[5547](tel:5547)) [Andrii Soldatenko] >>>>>>>>>> [AIRFLOW-[4884](tel:4884)] Roll up import_errors in RBAC UI >>>> (#[5516](tel:5516)) >>>>>>>>> [shuwen-zhang] >>>>>>>>>> [AIRFLOW-[4937](tel:4937)] Fix lodash security issue with = version >>>> below [4.17.13](tel:41713) >>>>>>>>>> (#[5572](tel:5572)) [Jarek Potiuk] >>>>>>>>>> [AIRFLOW-[4934](tel:4934)] Bump Flask to resolve Werkzeug = ProxyFix >>>> (#[5571](tel:5571)) >>>>>>>>> [Nathaniel >>>>>>>>>> Ritholtz] >>>>>>>>>> [AIRFLOW-[4934](tel:4934)] Fix ProxyFix due to Werkzeug = upgrade >>>> (#[5563](tel:5563)) >>>>>>>>> [Nathaniel >>>>>>>>>> Ritholtz] >>>>>>>>>> [AIRFLOW-[4136](tel:4136)] fix key_file of hook is = overwritten by >>>> SSHHook >>>>>>>>> connection >>>>>>>>>> (#[5558](tel:5558)) [eladkal] >>>>>>>>>> [AIRFLOW-XXX] Correct BaseSensorOperator docs = (#[5562](tel:5562)) >>>> [Matthew >>>>>>>>> Bowden] >>>>>>>>>> [AIRFLOW-[3495](tel:3495)] Validate one of query and = query_uri >>>> passed to >>>>>>>>>> DataProcSparkSqlOperator (#[5510](tel:5510)) [eladkal] >>>>>>>>>> [AIRFLOW-[4868](tel:4868)] Fix typo in = kubernetes/docker/build.sh >>>> (#[5505](tel:5505)) [Facu] >>>>>>>>>> [AIRFLOW-[4925](tel:4925)] Improve css style for Variables = Import >>>> file field >>>>>>>>> (#[5552](tel:5552)) >>>>>>>>>> [Felix Uellendall] >>>>>>>>>> [AIRFLOW-[4906](tel:4906)] Improve debugging for the >>>> SparkSubmitHook (#[5542](tel:5542)) >>>>>>>>> [Fokko >>>>>>>>>> Driesprong] >>>>>>>>>> [AIRFLOW-[4904](tel:4904)] Retrieve test config file from >>>> $AIRFLOW_TEST_CONFIG >>>>>>>>> (#[5540](tel:5540)) >>>>>>>>>> [Matthew Rastovac] >>>>>>>>>> [AIRFLOW-[4926](tel:4926)] Fix example dags where its = start_date >> is >>>>>>>>> datetime.utcnow() >>>>>>>>>> (#[5553](tel:5553)) [Felix Uellendall] >>>>>>>>>> [AIRFLOW-[4587](tel:4587)] Replace self.conn with = self.get_conn() >> in >>>>>>>>> AWSAthenaHook >>>>>>>>>> (#[5545](tel:5545)) [Bhavika Tekwani] >>>>>>>>>> [AIRFLOW-[1740](tel:1740)] Fix xcom creation and update via = UI >>>> (#[5530](tel:5530)) [Felix >>>>>>>>>> Uellendall] >>>>>>>>>> [AIRFLOW-[1740](tel:1740)] Fix xcom creation and update via = RBAC >> UI >>>> (#[5561](tel:5561)) >>>>>>>>> [Felix >>>>>>>>>> Uellendall] >>>>>>>>>> [AIRFLOW-[4920](tel:4920)] Use html.escape instead of = cgi.escape >> to >>>> fix >>>>>>>>>> DeprecationWarning (#[5551](tel:5551)) [Xtreak] >>>>>>>>>> [AIRFLOW-[4919](tel:4919)] DataProcJobBaseOperator >>>> dataproc_*_properties >>>>>>>>> templated >>>>>>>>>> (#[5555](tel:5555)) [eladkal] >>>>>>>>>> [AIRFLOW-[4900](tel:4900)] Resolve incompatible version of >> Werkzeug >>>> (#[5535](tel:5535)) >>>>>>>>> [Luqman >>>>>>>>>> Abdul Mushawwir] >>>>>>>>>> [AIRFLOW-[4478](tel:4478)] Lazily instantiate default = resources >>>> objects. (#[5259](tel:5259)) >>>>>>>>>> [Joshua Carp] >>>>>>>>>> [AIRFLOW-[4510](tel:4510)] Don't mutate default_args during = DAG >>>> initialization >>>>>>>>> (#[5277](tel:5277)) >>>>>>>>>> [Abhishek Ray] >>>>>>>>>> [AIRFLOW-[3360](tel:3360)] Make the DAGs search respect other >>>> querystring >>>>>>>>> parameters >>>>>>>>>> with url-search-params-polyfill for IE support = (#[5503](tel:5503)) >>>> [Alex] >>>>>>>>>> [AIRFLOW-[4896](tel:4896)] Make KubernetesExecutorConfig's = default >>>> args immutable >>>>>>>>>> (#[5534](tel:5534)) [Philippe Gagnon] >>>>>>>>>> [AIRFLOW-[4494](tel:4494)] Remove `shell=3DTrue` in = DaskExecutor >>>> (#[5273](tel:5273)) [MisLink] >>>>>>>>>> [AIRFLOW-[4890](tel:4890)] Fix Log link in TaskInstance's = View for >>>> Non-RBAC >>>>>>>>> (#[5525](tel:5525)) >>>>>>>>>> [Felix Uellendall] >>>>>>>>>> [AIRFLOW-[4892](tel:4892)] Fix connection creation via UIs >>>> (#[5527](tel:5527)) [Ash >>>>>>>>> Berlin-Taylor] >>>>>>>>>> [AIRFLOW-[4862](tel:4862)] Fix bug for earlier change to = allow >>>> using IP as >>>>>>>>> hostname >>>>>>>>>> (#[5513](tel:5513)) [Xiaodong] >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> Full Changelog since [1.10.3](tel:1103): >>>>>>>>>>=20 >>>>>>>>>> Airflow [1.10.4](tel:1104), - [2019-07-18](tel:20190718) >>>>>>>>>> ---------------------------- >>>>>>>>>>=20 >>>>>>>>>> New Features >>>>>>>>>> """""""""""" >>>>>>>>>> - [AIRFLOW-[4884](tel:4884)] Roll up import_errors in RBAC UI >>>> (#[5516](tel:5516)) >>>>>>>>>> - [AIRFLOW-[4871](tel:4871)] Allow creating DagRuns via RBAC = UI >>>> (#[5507](tel:5507)) >>>>>>>>>> - [AIRFLOW-[4591](tel:4591)] Make default_pool a real pool >>>> (#[5349](tel:5349)) >>>>>>>>>> - [AIRFLOW-[4844](tel:4844)] Add optional = is_paused_upon_creation >>>> argument to DAG >>>>>>>>>> (#[5473](tel:5473)) >>>>>>>>>> - [AIRFLOW-[4456](tel:4456)] Add sub-classable = BaseBranchOperator >>>> (#[5231](tel:5231)) >>>>>>>>>> - [AIRFLOW-[4343](tel:4343)] Show warning in UI if scheduler = is >> not >>>> running >>>>>>>>> (#[5127](tel:5127)) >>>>>>>>>> - [AIRFLOW-[4739](tel:4739)] Add ability to arbitrarily = define >>>> kubernetes worker >>>>>>>>> pod >>>>>>>>>> labels (#[5376](tel:5376)) >>>>>>>>>> - [AIRFLOW-[4348](tel:4348)] Add GCP console link in >>>> BigQueryOperator (#[5195](tel:5195)) >>>>>>>>>> - [AIRFLOW-[4306](tel:4306)] Global operator extra links >>>> (#[5094](tel:5094)) >>>>>>>>>> - [AIRFLOW-[4812](tel:4812)] Add batch images annotation >>>> (#[5433](tel:5433)) >>>>>>>>>> - [AIRFLOW-[4135](tel:4135)] Add Google Cloud Build operator = and >>>> hook (#[5251](tel:5251)) >>>>>>>>>> - [AIRFLOW-[4781](tel:4781)] Add the ability to specify ports = in >>>>>>>>> kubernetesOperator >>>>>>>>>> (#[5410](tel:5410)) >>>>>>>>>> - [AIRFLOW-[4521](tel:4521)] Pause dag also pause its subdags >>>> (#[5283](tel:5283)) >>>>>>>>>> - [AIRFLOW-[4738](tel:4738)] Enforce exampleinclude for = example >>>> DAGs (#[5375](tel:5375)) >>>>>>>>>> - [AIRFLOW-[4326](tel:4326)] Airflow AWS SQS Operator >>>> (#[5110](tel:5110)) >>>>>>>>>> - [AIRFLOW-[3729](tel:3729)] Support "DownwardAPI" in env >> variables >>>> for >>>>>>>>>> KubernetesPodOperator (#[4554](tel:4554)) >>>>>>>>>> - [AIRFLOW-[4585](tel:4585)] Implement Kubernetes Pod = Mutation >> Hook >>>> (#[5359](tel:5359)) >>>>>>>>>> - [AIRFLOW-[161](tel:161)] New redirect route and extra links >>>> (#[5059](tel:5059)) >>>>>>>>>> - [AIRFLOW-[4420](tel:4420)] Backfill respects = task_concurrency >>>> (#[5221](tel:5221)) >>>>>>>>>> - [AIRFLOW-[4147](tel:4147)] Add Operator to publish event to >> Redis >>>> (#[4967](tel:4967)) >>>>>>>>>> - [AIRFLOW-[3359](tel:3359)] Add option to pass customer >> encryption >>>> keys to >>>>>>>>> Dataproc >>>>>>>>>> (#[4200](tel:4200)) >>>>>>>>>> - [AIRFLOW-[4318](tel:4318)] Create Google Cloud Translate = Speech >>>> Operator >>>>>>>>> (#[5102](tel:5102)) >>>>>>>>>> - [AIRFLOW-[3960](tel:3960)] Adds Google Cloud Speech = operators >>>> (#[4780](tel:4780)) >>>>>>>>>> - [AIRFLOW-[1501](tel:1501)] Add = GoogleCloudStorageDeleteOperator >>>> (#[5230](tel:5230)) >>>>>>>>>> - [AIRFLOW-[3672](tel:3672)] Add support for Mongo DB DNS = Seedlist >>>> Connection >>>>>>>>> Format >>>>>>>>>> (#[4481](tel:4481)) >>>>>>>>>> - [AIRFLOW-[4397](tel:4397)] add integrations docs manually = for >> gcs >>>> sensors >>>>>>>>> (#[5204](tel:5204)) >>>>>>>>>> - [AIRFLOW-[4251](tel:4251)] Instrument DagRun schedule delay >>>> (#[5050](tel:5050)) >>>>>>>>>> - [AIRFLOW-[4118](tel:4118)] instrument DagRun duration >>>> (#[4946](tel:4946)) >>>>>>>>>> - [AIRFLOW-[4361](tel:4361)] Fix flaky >>>>>>>>> test_integration_run_dag_with_scheduler_failure >>>>>>>>>> (#[5182](tel:5182)) >>>>>>>>>> - [AIRFLOW-[4361](tel:4361)] Fix flaky >>>>>>>>> test_integration_run_dag_with_scheduler_failure >>>>>>>>>> (#[5140](tel:5140)) >>>>>>>>>> - [AIRFLOW-[4168](tel:4168)] Create Google Cloud Video >> Intelligence >>>> Operators >>>>>>>>> (#[4985](tel:4985)) >>>>>>>>>> - [AIRFLOW-[4397](tel:4397)] Add = GCSUploadSessionCompleteSensor >>>> (#[5166](tel:5166)) >>>>>>>>>> - [AIRFLOW-[4335](tel:4335)] Add default num_retries to GCP >>>> connection (#[5117](tel:5117)) >>>>>>>>>> - [AIRFLOW-[3808](tel:3808)] Add cluster_fields to = BigQueryHook's >>>>>>>>> create_empty_table >>>>>>>>>> (#[4654](tel:4654)) >>>>>>>>>> - [AIRFLOW-[4362](tel:4362)] Fix >> test_execution_limited_parallelism >>>> (#[5141](tel:5141)) >>>>>>>>>> - [AIRFLOW-[4307](tel:4307)] Backfill respects concurrency = limit >>>> (#[5128](tel:5128)) >>>>>>>>>> - [AIRFLOW-[4268](tel:4268)] Add = MsSqlToGoogleCloudStorageOperator >>>> (#[5077](tel:5077)) >>>>>>>>>> - [AIRFLOW-[4169](tel:4169)] Add Google Cloud Vision Detect >>>> Operators (#[4986](tel:4986)) >>>>>>>>>> - [AIRFLOW-XXX] Fix WS-[2019-0032](tel:20190032) >> (#[5384](tel:5384)) >>>>>>>>>> - [AIRFLOW-XXX] Fix CVE-[2019-11358](tel:201911358) >>>> (#[5197](tel:5197)) >>>>>>>>>> - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix >>>>>>>>> CVE-[2019-10906](tel:201910906) >>>>>>>>>> (#[5075](tel:5075)) >>>>>>>>>>=20 >>>>>>>>>> Improvement >>>>>>>>>> """"""""""" >>>>>>>>>> - [AIRFLOW-[4797](tel:4797)] Improve performance and = behaviour of >>>> zombie >>>>>>>>> detection >>>>>>>>>> (#[5511](tel:5511)) >>>>>>>>>> - [AIRFLOW-[4911](tel:4911)] Silence the FORBIDDEN errors = from the >>>>>>>>> KubernetesExecutor >>>>>>>>>> (#[5547](tel:5547)) >>>>>>>>>> - [AIRFLOW-[3495](tel:3495)] Validate one of query and = query_uri >>>> passed to >>>>>>>>>> DataProcSparkSqlOperator (#[5510](tel:5510)) >>>>>>>>>> - [AIRFLOW-[4925](tel:4925)] Improve css style for Variables >> Import >>>> file field >>>>>>>>> (#[5552](tel:5552)) >>>>>>>>>> - [AIRFLOW-[4906](tel:4906)] Improve debugging for the >>>> SparkSubmitHook (#[5542](tel:5542)) >>>>>>>>>> - [AIRFLOW-[4904](tel:4904)] unittest.cfg name and path can = be >>>> overriden by >>>>>>>>> setting >>>>>>>>>> $AIRFLOW_TEST_CONFIG (#[5540](tel:5540)) >>>>>>>>>> - [AIRFLOW-[4920](tel:4920)] Use html.escape instead of = cgi.escape >>>> to fix >>>>>>>>>> DeprecationWarning (#[5551](tel:5551)) >>>>>>>>>> - [AIRFLOW-[4919](tel:4919)] DataProcJobBaseOperator >>>> dataproc_*_properties >>>>>>>>> templated >>>>>>>>>> (#[5555](tel:5555)) >>>>>>>>>> - [AIRFLOW-[4478](tel:4478)] Lazily instantiate default = resources >>>> objects. >>>>>>>>> (#[5259](tel:5259)) >>>>>>>>>> - [AIRFLOW-[4564](tel:4564)] AzureContainerInstance bugfixes = and >>>> improvements >>>>>>>>> (#[5319](tel:5319)) >>>>>>>>>> - [AIRFLOW-[4237](tel:4237)] Including Try Number of Task in = Gantt >>>> Chart (#[5037](tel:5037)) >>>>>>>>>> - [AIRFLOW-[4862](tel:4862)] Allow directly using IP address = as >>>> hostname for >>>>>>>>> webserver >>>>>>>>>> logs (#[5501](tel:5501)) >>>>>>>>>> - [AIRFLOW-[4857](tel:4857)] Add templated fields to >>>> SlackWebhookOperator (#[5490](tel:5490)) >>>>>>>>>> - [AIRFLOW-[3502](tel:3502)] Add celery config option for = setting >>>> "pool" (#[4308](tel:4308)) >>>>>>>>>> - [AIRFLOW-[3217](tel:3217)] Button to toggle line wrapping = in log >>>> and code views >>>>>>>>>> (#[4277](tel:4277)) >>>>>>>>>> - [AIRFLOW-[4491](tel:4491)] Add a "Jump to end" button for = logs >>>> (#[5266](tel:5266)) >>>>>>>>>> - [AIRFLOW-[4422](tel:4422)] Pool utilization stats >>>> (#[5453](tel:5453)) >>>>>>>>>> - [AIRFLOW-[4805](tel:4805)] Add py_file as templated field = in >>>>>>>>> DataflowPythonOperator >>>>>>>>>> (#[5451](tel:5451)) >>>>>>>>>> - [AIRFLOW-[4838](tel:4838)] Surface Athena errors in >>>> AWSAthenaOperator (#[5467](tel:5467)) >>>>>>>>>> - [AIRFLOW-[4831](tel:4831)] conf.has_option no longer throws = if >>>> section is >>>>>>>>> missing. >>>>>>>>>> (#[5455](tel:5455)) >>>>>>>>>> - [AIRFLOW-[4829](tel:4829)] More descriptive exceptions for = EMR >>>> sensors (#[5452](tel:5452)) >>>>>>>>>> - [AIRFLOW-[4414](tel:4414)] AWSAthenaOperator: Push >>>> QueryExecutionID to XCom >>>>>>>>> (#[5276](tel:5276)) >>>>>>>>>> - [AIRFLOW-[4791](tel:4791)] add "schema" keyword arg to >>>> SnowflakeOperator >>>>>>>>> (#[5415](tel:5415)) >>>>>>>>>> - [AIRFLOW-[4759](tel:4759)] Don't error when marking = sucessful >> run >>>> as failed >>>>>>>>> (#[5435](tel:5435)) >>>>>>>>>> - [AIRFLOW-[4716](tel:4716)] Instrument dag loading time = duration >>>> (#[5350](tel:5350)) >>>>>>>>>> - [AIRFLOW-[3958](tel:3958)] Support list tasks as upstream = in >>>> chain (#[4779](tel:4779)) >>>>>>>>>> - [AIRFLOW-[4409](tel:4409)] Prevent task duration break by = null >>>> value (#[5178](tel:5178)) >>>>>>>>>> - [AIRFLOW-[4418](tel:4418)] Add "failed only" option to task >> modal >>>> (#[5193](tel:5193)) >>>>>>>>>> - [AIRFLOW-[4740](tel:4740)] Accept string ``end_date`` in = DAG >>>> default_args >>>>>>>>> (#[5381](tel:5381)) >>>>>>>>>> - [AIRFLOW-[4423](tel:4423)] Improve date handling in mysql = to gcs >>>> operator. >>>>>>>>> (#[5196](tel:5196)) >>>>>>>>>> - [AIRFLOW-[4447](tel:4447)] Display task duration as human >>>> friendly format in UI >>>>>>>>>> (#[5218](tel:5218)) >>>>>>>>>> - [AIRFLOW-[4377](tel:4377)] Remove needless object = conversion in >>>> DAG.owner() >>>>>>>>> (#[5144](tel:5144)) >>>>>>>>>> - [AIRFLOW-[4766](tel:4766)] Add autoscaling option for >>>>>>>>> DataprocClusterCreateOperator >>>>>>>>>> (#[5425](tel:5425)) >>>>>>>>>> - [AIRFLOW-[4795](tel:4795)] Upgrade alembic to latest = release. >>>> (#[5411](tel:5411)) >>>>>>>>>> - [AIRFLOW-[4793](tel:4793)] Add signature_name to mlengine >>>> operator (#[5417](tel:5417)) >>>>>>>>>> - [AIRFLOW-[3211](tel:3211)] Reattach to GCP Dataproc jobs = upon >>>> Airflow restart >>>>>>>>>> (#[4083](tel:4083)) >>>>>>>>>> - [AIRFLOW-[4750](tel:4750)] Log identified zombie task = instances >>>> (#[5389](tel:5389)) >>>>>>>>>> - [AIRFLOW-[3870](tel:3870)] STFPOperator: Update log level = and >>>> return value >>>>>>>>> (#[4355](tel:4355)) >>>>>>>>>> - [AIRFLOW-[4759](tel:4759)] Batch queries in set_state API. >>>> (#[5403](tel:5403)) >>>>>>>>>> - [AIRFLOW-[2737](tel:2737)] Restore original license header = to >>>>>>>>>> airflow.api.auth.backend.kerberos_auth >>>>>>>>>> - [AIRFLOW-[3635](tel:3635)] Fix incorrect logic in = detele_dag >>>> (introduced in >>>>>>>>> PR#[4406](tel:4406)) >>>>>>>>>> (#[4445](tel:4445)) >>>>>>>>>> - [AIRFLOW-[3599](tel:3599)] Removed Dagbag from delete dag >>>> (#[4406](tel:4406)) >>>>>>>>>> - [AIRFLOW-[4737](tel:4737)] Increase and document celery = queue >>>> name limit >>>>>>>>> (#[5383](tel:5383)) >>>>>>>>>> - [AIRFLOW-[4505](tel:4505)] Correct Tag ALL for PY3 >>>> (#[5275](tel:5275)) >>>>>>>>>> - [AIRFLOW-[4743](tel:4743)] Add environment variables = support to >>>> SSHOperator >>>>>>>>> (#[5385](tel:5385)) >>>>>>>>>> - [AIRFLOW-[4725](tel:4725)] Fix setup.py PEP[440](tel:440) & >>>> Sphinx-PyPI-upload dependency >>>>>>>>>> (#[5363](tel:5363)) >>>>>>>>>> - [AIRFLOW-[3370](tel:3370)] Add stdout output options to >>>> Elasticsearch task log >>>>>>>>>> handler (#[5048](tel:5048)) >>>>>>>>>> - [AIRFLOW-[4396](tel:4396)] Provide a link to external >>>> Elasticsearch logs in UI. >>>>>>>>>> (#[5164](tel:5164)) >>>>>>>>>> - [AIRFLOW-[1381](tel:1381)] Allow setting host temporary >> directory >>>> in >>>>>>>>> DockerOperator >>>>>>>>>> (#[5369](tel:5369)) >>>>>>>>>> - [AIRFLOW-[4598](tel:4598)] Task retries are not exhausted = for >> K8s >>>> executor >>>>>>>>> (#[5347](tel:5347)) >>>>>>>>>> - [AIRFLOW-[4218](tel:4218)] Support to Provide http args to >>>> K8executor while >>>>>>>>> calling >>>>>>>>>> k8 python client lib apis (#[5060](tel:5060)) >>>>>>>>>> - [AIRFLOW-[4159](tel:4159)] Add support for additional = static pod >>>> labels for >>>>>>>>>> K8sExecutor (#[5134](tel:5134)) >>>>>>>>>> - [AIRFLOW-[4720](tel:4720)] Allow comments in .airflowignore >>>> files. (#[5355](tel:5355)) >>>>>>>>>> - [AIRFLOW-[4486](tel:4486)] Add AWS IAM authenication in >> MySqlHook >>>> (#[5334](tel:5334)) >>>>>>>>>> - [AIRFLOW-[4417](tel:4417)] Add AWS IAM authenication for >>>> PostgresHook (#[5223](tel:5223)) >>>>>>>>>> - [AIRFLOW-[3990](tel:3990)] Compile regular expressions. >>>> (#[4813](tel:4813)) >>>>>>>>>> - [AIRFLOW-[4572](tel:4572)] Rename prepare_classpath() to >>>> prepare_syspath() >>>>>>>>> (#[5328](tel:5328)) >>>>>>>>>> - [AIRFLOW-[3869](tel:3869)] Raise consistent exception in >>>>>>>>>> AirflowConfigParser.getboolean (#[4692](tel:4692)) >>>>>>>>>> - [AIRFLOW-[4571](tel:4571)] Add headers to templated field = for >>>>>>>>> SimpleHttpOperator >>>>>>>>>> (#[5326](tel:5326)) >>>>>>>>>> - [AIRFLOW-[3867](tel:3867)] Rename GCP's subpackage >>>> (#[4690](tel:4690)) >>>>>>>>>> - [AIRFLOW-[3725](tel:3725)] Add private_key to bigquery_hook >>>> get_pandas_df >>>>>>>>> (#[4549](tel:4549)) >>>>>>>>>> - [AIRFLOW-[4546](tel:4546)] Upgrade google-cloud-bigtable. >>>> (#[5307](tel:5307)) >>>>>>>>>> - [AIRFLOW-[4519](tel:4519)] Optimise operator classname = sorting >> in >>>> views (#[5282](tel:5282)) >>>>>>>>>> - [AIRFLOW-[4503](tel:4503)] Support fully pig options >>>> (#[5271](tel:5271)) >>>>>>>>>> - [AIRFLOW-[4468](tel:4468)] add sql_alchemy_max_overflow >> parameter >>>> (#[5249](tel:5249)) >>>>>>>>>> - [AIRFLOW-[4467](tel:4467)] Add dataproc_jars to templated = fields >>>> in Dataproc >>>>>>>>> oper=E2=80=A6 >>>>>>>>>> (#[5248](tel:5248)) >>>>>>>>>> - [AIRFLOW-[4381](tel:4381)] Use get_direct_relative_ids get = task >>>> relatives >>>>>>>>> (#[5147](tel:5147)) >>>>>>>>>> - [AIRFLOW-[3624](tel:3624)] Add masterType parameter to >>>> MLEngineTrainingOperator >>>>>>>>>> (#[4428](tel:4428)) >>>>>>>>>> - [AIRFLOW-[3143](tel:3143)] Support Auto-Zone in >>>> DataprocClusterCreateOperator >>>>>>>>> (#[5169](tel:5169)) >>>>>>>>>> - [AIRFLOW-[3874](tel:3874)] Improve >>>> BigQueryHook.run_with_configuration's >>>>>>>>> location >>>>>>>>>> support (#[4695](tel:4695)) >>>>>>>>>> - [AIRFLOW-[4399](tel:4399)] Avoid duplicated = os.path.isfile() >>>> check in >>>>>>>>> models.dagbag >>>>>>>>>> (#[5165](tel:5165)) >>>>>>>>>> - [AIRFLOW-[4031](tel:4031)] Allow for key pair auth in = snowflake >>>> hook (#[4875](tel:4875)) >>>>>>>>>> - [AIRFLOW-[3901](tel:3901)] add role as optional config = parameter >>>> for >>>>>>>>> SnowflakeHook >>>>>>>>>> (#[4721](tel:4721)) >>>>>>>>>> - [AIRFLOW-[3455](tel:3455)] add region in snowflake = connector >>>> (#[4285](tel:4285)) >>>>>>>>>> - [AIRFLOW-[4073](tel:4073)] add template_ext for AWS Athena >>>> operator (#[4907](tel:4907)) >>>>>>>>>> - [AIRFLOW-[4093](tel:4093)] AWSAthenaOperator: Throw = exception if >>>> job >>>>>>>>>> failed/cancelled/reach max retries (#[4919](tel:4919)) >>>>>>>>>> - [AIRFLOW-[4356](tel:4356)] Add extra RuntimeEnvironment = keys to >>>> DataFlowHook >>>>>>>>> (#[5149](tel:5149)) >>>>>>>>>> - [AIRFLOW-[4337](tel:4337)] Fix docker-compose deprecation >> warning >>>> in CI (#[5119](tel:5119)) >>>>>>>>>> - [AIRFLOW-[3603](tel:3603)] QuboleOperator: Remove = SQLCommand >> from >>>> SparkCmd >>>>>>>>>> documentation (#[4411](tel:4411)) >>>>>>>>>> - [AIRFLOW-[4328](tel:4328)] Fix link to task instances from = Pool >>>> page (#[5124](tel:5124)) >>>>>>>>>> - [AIRFLOW-[4255](tel:4255)] Make GCS Hook Backwards = compatible >>>> (#[5089](tel:5089)) >>>>>>>>>> - [AIRFLOW-[4103](tel:4103)] Allow uppercase letters in = dataflow >>>> job names >>>>>>>>> (#[4925](tel:4925)) >>>>>>>>>> - [AIRFLOW-[4255](tel:4255)] Replace Discovery based api with >>>> client based for >>>>>>>>> GCS >>>>>>>>>> (#[5054](tel:5054)) >>>>>>>>>> - [AIRFLOW-[4311](tel:4311)] Remove sleep in localexecutor >>>> (#[5096](tel:5096)) >>>>>>>>>> - [AIRFLOW-[2836](tel:2836)] Minor >>>> improvement-contrib.sensors.FileSensor (#[3674](tel:3674)) >>>>>>>>>> - [AIRFLOW-[4104](tel:4104)] Add type annotations to common >>>> classes. (#[4926](tel:4926)) >>>>>>>>>> - [AIRFLOW-[3910](tel:3910)] Raise exception explicitly in >>>> Connection.get_hook() >>>>>>>>>> (#[4728](tel:4728)) >>>>>>>>>> - [AIRFLOW-[3322](tel:3322)] Update QuboleHook to fetch args >>>> dynamically from >>>>>>>>> qds_sdk >>>>>>>>>> (#[4165](tel:4165)) >>>>>>>>>> - [AIRFLOW-[4565](tel:4565)] instrument celery executor >>>> (#[5321](tel:5321)) >>>>>>>>>> - [AIRFLOW-[4573](tel:4573)] Import airflow_local_settings = after >>>>>>>>> prepare_classpath >>>>>>>>>> (#[5330](tel:5330)) >>>>>>>>>> - [AIRFLOW-[4448](tel:4448)] Don't bake ENV and _cmd into tmp >>>> config for non-sudo >>>>>>>>>> (#[4050](tel:4050)) >>>>>>>>>> - [AIRFLOW-[4295](tel:4295)] Make ``method`` attribute case >>>> insensitive in >>>>>>>>> HttpHook >>>>>>>>>> (#[5313](tel:5313)) >>>>>>>>>> - [AIRFLOW-[3703](tel:3703)] Add dnsPolicy option for >>>> KubernetesPodOperator >>>>>>>>> (#[4520](tel:4520)) >>>>>>>>>> - [AIRFLOW-[3057](tel:3057)] add prev_*_date_success to = template >>>> context (#[5372](tel:5372)) >>>>>>>>>> - [AIRFLOW-[4336](tel:4336)] Stop showing entire GCS files = bytes >> in >>>> log for >>>>>>>>>> gcs_download_operator (#[5151](tel:5151)) >>>>>>>>>> - [AIRFLOW-[4528](tel:4528)] Cancel DataProc task on timeout >>>> (#[5293](tel:5293)) >>>>>>>>>>=20 >>>>>>>>>> Bug fixes >>>>>>>>>> """"""""" >>>>>>>>>> - [AIRFLOW-[4934](tel:4934)] Fix ProxyFix due to Werkzeug = upgrade >>>> (#[5563](tel:5563)) (#[5571](tel:5571)) >>>>>>>>>> - [AIRFLOW-[4136](tel:4136)] fix key_file of hook is = overwritten >> by >>>> SSHHook >>>>>>>>> connection >>>>>>>>>> (#[5558](tel:5558)) >>>>>>>>>> - [AIRFLOW-[4587](tel:4587)] Replace self.conn with >> self.get_conn() >>>> in >>>>>>>>> AWSAthenaHook >>>>>>>>>> (#[5545](tel:5545)) >>>>>>>>>> - [AIRFLOW-[1740](tel:1740)] Fix xcom creation and update via = UI >>>> (#[5530](tel:5530)) (#[5531](tel:5531)) >>>>>>>>>> - [AIRFLOW-[4900](tel:4900)] Resolve incompatible version of >>>> Werkzeug (#[5535](tel:5535)) >>>>>>>>>> - [AIRFLOW-[4510](tel:4510)] Don't mutate default_args during = DAG >>>> initialization >>>>>>>>>> (#[5277](tel:5277)) >>>>>>>>>> - [AIRFLOW-[3360](tel:3360)] Make the DAGs search respect = other >>>> querystring >>>>>>>>> parameters >>>>>>>>>> with url-search-params-polyfill for IE support = (#[5503](tel:5503)) >>>>>>>>>> - [AIRFLOW-[4896](tel:4896)] Make KubernetesExecutorConfig's >>>> default args >>>>>>>>> immutable >>>>>>>>>> (#[5534](tel:5534)) >>>>>>>>>> - [AIRFLOW-[4494](tel:4494)] Remove ``shell=3DTrue`` in = DaskExecutor >>>> (#[5273](tel:5273)) >>>>>>>>>> - [AIRFLOW-[4890](tel:4890)] Fix Log link in TaskInstance's = View >>>> for Non-RBAC >>>>>>>>> (#[5525](tel:5525)) >>>>>>>>>> - [AIRFLOW-[4892](tel:4892)] Fix connection creation via UIs >>>> (#[5527](tel:5527)) >>>>>>>>>> - [AIRFLOW-[4406](tel:4406)] Fix a method name typo: >>>> NullFernet.decrpyt to >>>>>>>>> decrypt >>>>>>>>>> (#[5509](tel:5509)) >>>>>>>>>> - [AIRFLOW-[4849](tel:4849)] Add gcp_conn_id to = cloudsqldatabehook >>>> class to use >>>>>>>>>> correctly CloudSqlProxyRunner class (#[5478](tel:5478)) >>>>>>>>>> - [AIRFLOW-[4769](tel:4769)] Pass gcp_conn_id to BigtableHook >>>> (#[5445](tel:5445)) >>>>>>>>>> - [AIRFLOW-[4524](tel:4524)] Fix incorrect field names in = view for >>>> Mark >>>>>>>>>> Success/Failure (#[5486](tel:5486)) >>>>>>>>>> - [AIRFLOW-[3671](tel:3671)] Remove arg ``replace`` of >>>> MongoToS3Operator from >>>>>>>>>> ``kwargs`` (#[4480](tel:4480)) >>>>>>>>>> - [AIRFLOW-[4845](tel:4845)] Fix bug where runAsUser 0 = doesn't get >>>> set in k8s >>>>>>>>> security >>>>>>>>>> context (#[5474](tel:5474)) >>>>>>>>>> - [AIRFLOW-[4354](tel:4354)] Fix exception in "between" date >> filter >>>> in classic UI >>>>>>>>>> (#[5480](tel:5480)) >>>>>>>>>> - [AIRFLOW-[4587](tel:4587)] Replace self.conn with >> self.get_conn() >>>> in >>>>>>>>> AWSAthenaHook >>>>>>>>>> (#[5462](tel:5462)) >>>>>>>>>> - [AIRFLOW-[4516](tel:4516)] K8s runAsUser and fsGroup cannot = be >>>> strings (#[5429](tel:5429)) >>>>>>>>>> - [AIRFLOW-[4298](tel:4298)] Stop Scheduler repeatedly = warning >>>> "connection >>>>>>>>>> invalidated" (#[5470](tel:5470)) >>>>>>>>>> - [AIRFLOW-[4559](tel:4559)] JenkinsJobTriggerOperator bugfix >>>> (#[5318](tel:5318)) >>>>>>>>>> - [AIRFLOW-[4841](tel:4841)] Pin Sphinx AutoApi to >> [1.0.0](tel:100) >>>> (#[5468](tel:5468)) >>>>>>>>>> - [AIRFLOW-[4479](tel:4479)] Include s3_overwrite kwarg in >>>> load_bytes method >>>>>>>>> (#[5312](tel:5312)) >>>>>>>>>> - [AIRFLOW-[3746](tel:3746)] Fix DockerOperator missing = container >>>> exit (#[4583](tel:4583)) >>>>>>>>>> - [AIRFLOW-[4233](tel:4233)] Remove Template Extension from = Bq to >>>> GCS Operator >>>>>>>>> (#[5456](tel:5456)) >>>>>>>>>> - >>>>=20 >> = [AIRFLOW-[2141](tel:2141)][AIRFLOW-[3157](tel:3157)][AIRFLOW-[4170](tel:41= 70)] >>>> Serialize non-str value >>>>>>>>> by >>>>>>>>>> JSON when importing Variables (#[4991](tel:4991)) >>>>>>>>>> - [AIRFLOW-[4826](tel:4826)] Remove warning from ``airflow >>>> resetdb`` command >>>>>>>>> (#[5447](tel:5447)) >>>>>>>>>> - [AIRFLOW-[4148](tel:4148)] Fix editing DagRuns when = clicking >>>> state column >>>>>>>>> (#[5436](tel:5436)) >>>>>>>>>> - [AIRFLOW-[4455](tel:4455)] dag_details broken for subdags = in >> RBAC >>>> UI (#[5234](tel:5234)) >>>>>>>>>> - [AIRFLOW-[2955](tel:2955)] Fix kubernetes pod operator to = set >>>> requests and >>>>>>>>> limits on >>>>>>>>>> task pods (#[4551](tel:4551)) >>>>>>>>>> - [AIRFLOW-[4459](tel:4459)] Fix wrong DAG count in /home = page >> when >>>> DAG count is >>>>>>>>> zero >>>>>>>>>> (#[5235](tel:5235)) >>>>>>>>>> - [AIRFLOW-[3876](tel:3876)] AttributeError: module = 'distutils' >> has >>>> no attribute >>>>>>>>> 'util' >>>>>>>>>> - [AIRFLOW-[4146](tel:4146)] Fix CgroupTaskRunner errors >>>> (#[5224](tel:5224)) >>>>>>>>>> - [AIRFLOW-[4524](tel:4524)] Fix bug with "Ignore \*" toggles = in >>>> RBAC mode >>>>>>>>> (#[5378](tel:5378)) >>>>>>>>>> - [AIRFLOW-[4765](tel:4765)] Fix DataProcPigOperator execute >> method >>>> (#[5426](tel:5426)) >>>>>>>>>> - [AIRFLOW-[4798](tel:4798)] obviate interdependencies for = dagbag >>>> and TI tests >>>>>>>>> (#[5422](tel:5422)) >>>>>>>>>> - [AIRFLOW-[4800](tel:4800)] fix GKEClusterHook ctor calls >>>> (#[5424](tel:5424)) >>>>>>>>>> - [AIRFLOW-[4799](tel:4799)] don't mutate self.env in = BashOperator >>>> execute method >>>>>>>>>> (#[5421](tel:5421)) >>>>>>>>>> - [AIRFLOW-[4393](tel:4393)] Add retry logic when fetching = pod >>>> status and/or logs >>>>>>>>> in >>>>>>>>>> KubernetesPodOperator (#[5284](tel:5284)) >>>>>>>>>> - [AIRFLOW-[4174](tel:4174)] Fix HttpHook run with backoff >>>> (#[5213](tel:5213)) >>>>>>>>>> - [AIRFLOW-[4463](tel:4463)] Handle divide-by-zero errors in = short >>>> retry >>>>>>>>> intervals >>>>>>>>>> (#[5243](tel:5243)) >>>>>>>>>> - [AIRFLOW-[2614](tel:2614)] Speed up trigger_dag API call = when >>>> lots of DAGs in >>>>>>>>> system >>>>>>>>>> - [AIRFLOW-[4756](tel:4756)] add ti.state to ti.start_date as >>>> criteria for gantt >>>>>>>>>> (#[5399](tel:5399)) >>>>>>>>>> - [AIRFLOW-[4760](tel:4760)] Fix zip-packaged DAGs = disappearing >>>> from DagBag when >>>>>>>>>> reloaded (#[5404](tel:5404)) >>>>>>>>>> - [AIRFLOW-[4731](tel:4731)] Fix GCS hook with >>>> google-storage-client [1.16](tel:116) (#[5368](tel:5368)) >>>>>>>>>> - [AIRFLOW-[3506](tel:3506)] use match_phrase to query log_id = in >>>> elasticsearch >>>>>>>>> (#[4342](tel:4342)) >>>>>>>>>> - [AIRFLOW-[4084](tel:4084)] fix ElasticSearch log download >>>> (#[5177](tel:5177)) >>>>>>>>>> - [AIRFLOW-[4501](tel:4501)] Register pendulum datetime = converter >>>> for >>>>>>>>> sqla+pymysql >>>>>>>>>> (#[5190](tel:5190)) >>>>>>>>>> - [AIRFLOW-[986](tel:986)] HiveCliHook ignores 'proxy_user' = value >>>> in a >>>>>>>>> connection's >>>>>>>>>> extra parameter (#[5305](tel:5305)) >>>>>>>>>> - [AIRFLOW-[4442](tel:4442)] fix hive_tblproperties in >>>> HiveToDruidTransfer >>>>>>>>> (#[5211](tel:5211)) >>>>>>>>>> - [AIRFLOW-[4557](tel:4557)] Add gcp_conn_id parameter to >>>> get_sqlproxy_runner() >>>>>>>>> of >>>>>>>>>> CloudSqlDatabaseHook (#[5314](tel:5314)) >>>>>>>>>> - [AIRFLOW-[4545](tel:4545)] Upgrade FAB to latest version >>>> (#[4955](tel:4955)) >>>>>>>>>> - [AIRFLOW-[4492](tel:4492)] Change Dataproc Cluster = operators to >>>> poll Operations >>>>>>>>>> (#[5269](tel:5269)) >>>>>>>>>> - [AIRFLOW-[4452](tel:4452)] Webserver and Scheduler keep = crashing >>>> because of >>>>>>>>>> slackclient update (#[5225](tel:5225)) >>>>>>>>>> - [AIRFLOW-[4450](tel:4450)] Fix request arguments in >>>> has_dag_access (#[5220](tel:5220)) >>>>>>>>>> - [AIRFLOW-[4434](tel:4434)] Support Impala with the >>>> HiveServer2Hook (#[5206](tel:5206)) >>>>>>>>>> - [AIRFLOW-[3449](tel:3449)] Write local dag parsing logs = when >>>> remote logging >>>>>>>>> enabled. >>>>>>>>>> (#[5175](tel:5175)) >>>>>>>>>> - [AIRFLOW-[4300](tel:4300)] Fix graph modal call when DAG = has not >>>> yet run >>>>>>>>> (#[5185](tel:5185)) >>>>>>>>>> - [AIRFLOW-[4401](tel:4401)] Use managers for Queue >> synchronization >>>> (#[5200](tel:5200)) >>>>>>>>>> - [AIRFLOW-[3626](tel:3626)] Fixed triggering DAGs contained >> within >>>> zip files >>>>>>>>> (#[4439](tel:4439)) >>>>>>>>>> - [AIRFLOW-[3720](tel:3720)] Fix missmatch while comparing = GCS and >>>> S3 files >>>>>>>>> (#[4766](tel:4766)) >>>>>>>>>> - [AIRFLOW-[4403](tel:4403)] search by ``dag_id`` or = ``owners`` in >>>> UI (#[5184](tel:5184)) >>>>>>>>>> - [AIRFLOW-[4308](tel:4308)] Fix TZ-loop around DST on python = 3.6+ >>>> (#[5095](tel:5095)) >>>>>>>>>> - [AIRFLOW-[4324](tel:4324)] fix DAG fuzzy search in RBAC UI >>>> (#[5131](tel:5131)) >>>>>>>>>> - [AIRFLOW-[4297](tel:4297)] Temporary hot fix on = manage_slas() >> for >>>> [1.10.4](tel:1104) >>>>>>>>> release >>>>>>>>>> (#[5150](tel:5150)) >>>>>>>>>> - [AIRFLOW-[4299](tel:4299)] Upgrade to Celery = [4.3.0](tel:430) to >>>> fix crashing workers >>>>>>>>> (#[5116](tel:5116)) >>>>>>>>>> - [AIRFLOW-[4291](tel:4291)] Correctly render doc_md in DAG = graph >>>> page (#[5121](tel:5121)) >>>>>>>>>> - [AIRFLOW-[4310](tel:4310)] Fix incorrect link on Dag = Details >> page >>>> (#[5122](tel:5122)) >>>>>>>>>> - [AIRFLOW-[4331](tel:4331)] Correct filter for Null-state = runs >>>> from Dag Detail >>>>>>>>> page >>>>>>>>>> (#[5123](tel:5123)) >>>>>>>>>> - [AIRFLOW-[4294](tel:4294)] Fix missing dag & task runs in = UI >>>> dag_id contains a >>>>>>>>> dot >>>>>>>>>> (#[5111](tel:5111)) >>>>>>>>>> - [AIRFLOW-[4332](tel:4332)] Upgrade sqlalchemy to remove = security >>>> Vulnerability >>>>>>>>>> (#[5113](tel:5113)) >>>>>>>>>> - [AIRFLOW-[4312](tel:4312)] Add template_fields & = template_ext to >>>>>>>>> BigQueryCheckO=E2=80=A6 >>>>>>>>>> (#[5097](tel:5097)) >>>>>>>>>> - [AIRFLOW-[4293](tel:4293)] Fix downgrade in >>>>>>>>>> d4ecb8fbee3_add_schedule_interval_to_dag.py = (#[5086](tel:5086)) >>>>>>>>>> - [AIRFLOW-[4267](tel:4267)] Fix TI duration in Graph View >>>> (#[5071](tel:5071)) >>>>>>>>>> - [AIRFLOW-[4163](tel:4163)] IntervalCheckOperator supports >>>> relative diff and not >>>>>>>>>> ignore 0 (#[4983](tel:4983)) >>>>>>>>>> - [AIRFLOW-[3938](tel:3938)] QuboleOperator Fixes and Support = for >>>> SqlCommand >>>>>>>>> (#[4832](tel:4832)) >>>>>>>>>> - [AIRFLOW-[2903](tel:2903)] Change default owner to = ``airflow`` >>>> (#[4151](tel:4151)) >>>>>>>>>> - [aIRFLOW-[4136](tel:4136)] Fix overwrite of key_file by >>>> constructor (#[5155](tel:5155)) >>>>>>>>>> - [AIRFLOW-[3241](tel:3241)] Remove Invalid template ext in = GCS >>>> Sensors (#[4076](tel:4076)) >>>>>>>>>>=20 >>>>>>>>>> Misc/Internal >>>>>>>>>> """"""""""""" >>>>>>>>>> - [AIRFLOW-[4937](tel:4937)] Fix lodash security issue with >> version >>>> below [4.17.13](tel:41713) >>>>>>>>>> (#[5572](tel:5572)) (used only in build-pipeline, not = runtime) >>>>>>>>>> - [AIRFLOW-[4868](tel:4868)] Fix typo in >> kubernetes/docker/build.sh >>>> (#[5505](tel:5505)) >>>>>>>>>> - [AIRFLOW-[4211](tel:4211)] Add tests for WebHDFSHook >>>> (#[5015](tel:5015)) >>>>>>>>>> - [AIRFLOW-[4320](tel:4320)] Add tests for >>>> SegmentTrackEventOperator (#[5104](tel:5104)) >>>>>>>>>> - [AIRFLOW-[4319](tel:4319)] Add tests for Bigquery related >>>> Operators (#[5101](tel:5101)) >>>>>>>>>> - [AIRFLOW-[4014](tel:4014)] Change DatastoreHook and add = tests >>>> (#[4842](tel:4842)) >>>>>>>>>> - [AIRFLOW-[4322](tel:4322)] Add test for VerticaOperator >>>> (#[5107](tel:5107)) >>>>>>>>>> - [AIRFLOW-[4323](tel:4323)] Add 2 tests for WinRMOperator >>>> (#[5108](tel:5108)) >>>>>>>>>> - [AIRFLOW-[3677](tel:3677)] Improve CheckOperator test = coverage >>>> (#[4756](tel:4756)) >>>>>>>>>> - [AIRFLOW-[4659](tel:4659)] Fix pylint problems for api = module >>>> (#[5398](tel:5398)) >>>>>>>>>> - [AIRFLOW-[4358](tel:4358)] Speed up test_jobs by not = running >>>> tasks (#[5162](tel:5162)) >>>>>>>>>> - [AIRFLOW-[4394](tel:4394)] Don't test behaviour of = BackfillJob >>>> from CLI tests >>>>>>>>> (#[5160](tel:5160)) >>>>>>>>>> - [AIRFLOW-[3471](tel:3471)] Move XCom out of models.py >>>> (#[4629](tel:4629)) >>>>>>>>>> - [AIRFLOW-[4379](tel:4379)] Remove duplicate code & Add >> validation >>>> in >>>>>>>>> gcs_to_gcs.py >>>>>>>>>> (#[5145](tel:5145)) >>>>>>>>>> - [AIRFLOW-[4259](tel:4259)] Move models out of models.py >>>> (#[5056](tel:5056)) >>>>>>>>>> - [AIRFLOW-XXX] Speed up building of Cassanda module on = Travis >>>>>>>>> (#[5233](tel:5233)) >>>>>>>>>> - [AIRFLOW-[4535](tel:4535)] Break jobs.py into multiple = files >>>> (#[5303](tel:5303)) >>>>>>>>>> - [AIRFLOW-[1464](tel:1464)] Batch update task_instance state >>>> (#[5323](tel:5323)) >>>>>>>>>> - [AIRFLOW-[4554](tel:4554)] Test for sudo command, add some = other >>>> test docs >>>>>>>>> (#[5310](tel:5310)) >>>>>>>>>> - [AIRFLOW-[4419](tel:4419)] Refine concurrency check in = scheduler >>>> (#[5194](tel:5194)) >>>>>>>>>> - [AIRFLOW-[4269](tel:4269)] Minor acceleration of >>>> jobs._process_task_instances() >>>>>>>>>> (#[5076](tel:5076)) >>>>>>>>>> - [AIRFLOW-[4341](tel:4341)] Remove ``View.render()`` already >>>> exists in >>>>>>>>> fab.BaseView >>>>>>>>>> (#[5125](tel:5125)) >>>>>>>>>> - [AIRFLOW-[4342](tel:4342)] Use @cached_property instead of >>>> re-implementing it >>>>>>>>> each >>>>>>>>>> time (#[5126](tel:5126)) >>>>>>>>>> - [AIRFLOW-[4256](tel:4256)] Remove noqa from migrations >>>> (#[5055](tel:5055)) >>>>>>>>>> - [AIRFLOW-[4034](tel:4034)] Remove unnecessary string = formatting >>>> with >>>>>>>>> ``**locals()`` >>>>>>>>>> (#[4861](tel:4861)) >>>>>>>>>> - [AIRFLOW-[3944](tel:3944)] Remove code smells >> (#[4762](tel:4762)) >>>>>>>>>>=20 >>>>>>>>>> Doc-only changes >>>>>>>>>> """""""""""""""" >>>>>>>>>> - [AIRFLOW-XXX] Correct BaseSensorOperator docs >> (#[5562](tel:5562)) >>>>>>>>>> - [AIRFLOW-[4926](tel:4926)] Fix example dags where its = start_date >>>> is >>>>>>>>>> datetime.utcnow() (#[5553](tel:5553)) >>>>>>>>>> - [AIRFLOW-[4860](tel:4860)] Remove Redundant Information in >>>> Example Dags (#[5497](tel:5497)) >>>>>>>>>> - [AIRFLOW-[4767](tel:4767)] Fix errors in the documentation = of >>>> Dataproc Operator >>>>>>>>>> (#[5487](tel:5487)) >>>>>>>>>> - [AIRFLOW-[1684](tel:1684)] Branching based on XCom variable >>>> (Docs) (#[4365](tel:4365)) >>>>>>>>>> - [AIRFLOW-[3341](tel:3341)] FAQ return DAG object example >>>> (#[4605](tel:4605)) >>>>>>>>>> - [AIRFLOW-[4433](tel:4433)] Add missing type in = DockerOperator >> doc >>>> string >>>>>>>>> (#[5205](tel:5205)) >>>>>>>>>> - [AIRFLOW-[4321](tel:4321)] Replace incorrect info of Max = Size >>>> limit of GCS >>>>>>>>> Object >>>>>>>>>> Size (#[5106](tel:5106)) >>>>>>>>>> - [AIRFLOW-XXX] Add information about user list >> (#[5341](tel:5341)) >>>>>>>>>> - [AIRFLOW-XXX] Clarify documentation related to autodetect >>>> parameter >>>>>>>>> in >>>>>>>>>> GCS_to_BQ Op (#[5294](tel:5294)) >>>>>>>>>> - [AIRFLOW-XXX] Remove mention of pytz compatibility from = timezone >>>>>>>>>> documentation (#[5316](tel:5316)) >>>>>>>>>> - [AIRFLOW-XXX] Add missing docs for >>>> GoogleCloudStorageDeleteOperator >>>>>>>>>> (#[5274](tel:5274)) >>>>>>>>>> - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP >> connection >>>>>>>>>> (#[5242](tel:5242)) >>>>>>>>>> - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators >>>> (#[5192](tel:5192)) >>>>>>>>>> - [AIRFLOW-XXX] Link to correct class for timedelta in = macros.rst >>>>>>>>> (#[5226](tel:5226)) >>>>>>>>>> - [AIRFLOW-XXX] Add Kamil as committer (#[5216](tel:5216)) >>>>>>>>>> - [AIRFLOW-XXX] Add Joshua and Kevin as committer >>>> (#[5207](tel:5207)) >>>>>>>>>> - [AIRFLOW-XXX] Reduce log spam in tests (#[5174](tel:5174)) >>>>>>>>>> - [AIRFLOW-XXX] Speed up tests for PythonSensor >> (#[5158](tel:5158)) >>>>>>>>>> - [AIRFLOW-XXX] Add Bas Harenslak to committer list >>>> (#[5157](tel:5157)) >>>>>>>>>> - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list >>>> (#[5132](tel:5132)) >>>>>>>>>> - [AIRFLOW-XXX] Update docstring for SchedulerJob >>>> (#[5105](tel:5105)) >>>>>>>>>> - [AIRFLOW-XXX] Fix docstrings for >>>>>>>>> CassandraToGoogleCloudStorageOperator >>>>>>>>>> (#[5103](tel:5103)) >>>>>>>>>> - [AIRFLOW-XXX] update SlackWebhookHook and = SlackWebhookOperator >>>>>>>>> docstring >>>>>>>>>> (#[5074](tel:5074)) >>>>>>>>>> - [AIRFLOW-XXX] Ignore python files under node_modules in = docs >>>>>>>>> (#[5063](tel:5063)) >>>>>>>>>> - [AIRFLOW-XXX] Build a universal wheel with LICNESE files >>>> (#[5052](tel:5052)) >>>>>>>>>> - [AIRFLOW-XXX] Fix docstrings of SQSHook (#[5099](tel:5099)) >>>>>>>>>> - [AIRFLOW-XXX] Use Py3.7 on readthedocs >>>>>>>>>> - [AIRFLOW-[4446](tel:4446)] Fix typos (#[5217](tel:5217)) >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Best regards, >>>>>>=20 >>>>>> Andrii Soldatenko >>>>>> Python Developer >>>>>>=20 >>>>>> skype: andrii.soldatenko >>>>>> andrii.soldatenko@gmail.com >>>>>> http://asoldatenko.com >>>=20 >>>=20 >>>=20 >>> -- >>> *Kaxil Naik* >>> *Big Data Consultant | DevOps Data Engineer* >>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & = Neo4j >>> Developer >>> *LinkedIn*: https://www.linkedin.com/in/kaxil >>=20 >>=20