Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4A701200D31 for ; Sat, 4 Nov 2017 20:42:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 48E41160BE9; Sat, 4 Nov 2017 19:42:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 67CA41609EE for ; Sat, 4 Nov 2017 20:42:05 +0100 (CET) Received: (qmail 77261 invoked by uid 500); 4 Nov 2017 19:42:04 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 77252 invoked by uid 99); 4 Nov 2017 19:42:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Nov 2017 19:42:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BB224180892 for ; Sat, 4 Nov 2017 19:42:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.201 X-Spam-Level: X-Spam-Status: No, score=-99.201 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Wmcit8p-1XwH for ; Sat, 4 Nov 2017 19:42:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 3D16B60E35 for ; Sat, 4 Nov 2017 19:42:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6ADD8E061F for ; Sat, 4 Nov 2017 19:42:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2743F241A0 for ; Sat, 4 Nov 2017 19:42:00 +0000 (UTC) Date: Sat, 4 Nov 2017 19:42:00 +0000 (UTC) From: "Dmytro Kulyk (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AIRFLOW-1783) DAG created from function become unavailable in UI sporadically MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sat, 04 Nov 2017 19:42:06 -0000 [ https://issues.apache.org/jira/browse/AIRFLOW-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmytro Kulyk updated AIRFLOW-1783: ---------------------------------- Description: I have a DAG created from function (see codescript and attachment below) After uploading to server it become visible and can be executed by scheduler or manually [^Capture_before.JPG] [^Capture_tree.JPG] However, in case of any activity upon this object (refresh etc) there is an "kaboom error" [^Capture_error.JPG] [^Capture_after.JPG] {code} ------------------------------------------------------------------------------- Node: uat01.airflow.retail ------------------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 69, in inner return self._run_view(f, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 368, in _run_view return fn(self, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 755, in decorated_view return func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 219, in view_func return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 125, in wrapper return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line 1161, in tree base_date = dag.latest_execution_date or datetime.now() AttributeError: 'NoneType' object has no attribute 'latest_execution_date' {code} In case when DAG created directly (without function) - everything works as a charm cube_update.py {code} from datetime import datetime from lib.cube_update import cube_updater env = 'sit' start_date = datetime(2017, 7, 1) schedule_interval = '0 8 * * *' dag = cube_updater(env, start_date, schedule_interval) {code} lib/__init__.py - empty one lib/cube_update.py {code} from airflow import DAG ... def cube_updater(env, start_date, schedule_interval): ... dag = DAG( dag_id = dag_id, default_args = default_args, description = 'OLAP Cube updater', start_date = start_date, schedule_interval = schedule_interval, max_active_runs = 6, catchup=True) .... return dag {code} was: I have a DAG created from function (see codescript and attachment below) After uploading to server it become visible and can be executed by scheduler or manually [^Capture_before.JPG] [^Capture_tree.JPG] However, in case of any activity upon this object (refresh etc) there is an "kaboom error" [^Capture_error.JPG] [^Capture_after.JPG] {code} ------------------------------------------------------------------------------- Node: uat01.airflow.retail ------------------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 69, in inner return self._run_view(f, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 368, in _run_view return fn(self, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 755, in decorated_view return func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 219, in view_func return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 125, in wrapper return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line 1161, in tree base_date = dag.latest_execution_date or datetime.now() AttributeError: 'NoneType' object has no attribute 'latest_execution_date' {code} cube_update.py {code} from datetime import datetime from lib.cube_update import cube_updater env = 'sit' start_date = datetime(2017, 7, 1) schedule_interval = '0 8 * * *' dag = cube_updater(env, start_date, schedule_interval) {code} lib/__init__.py - empty one lib/cube_update.py {code} from airflow import DAG ... def cube_updater(env, start_date, schedule_interval): ... dag = DAG( dag_id = dag_id, default_args = default_args, description = 'OLAP Cube updater', start_date = start_date, schedule_interval = schedule_interval, max_active_runs = 6, catchup=True) .... return dag {code} > DAG created from function become unavailable in UI sporadically > --------------------------------------------------------------- > > Key: AIRFLOW-1783 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1783 > Project: Apache Airflow > Issue Type: Bug > Components: DAG, scheduler, ui, webserver > Affects Versions: 1.8.1, 1.8.2 > Environment: Ubuntu 16.04.3 LTS > Python 2.7.12 > CeleryExecutor: 2-nodes cluster > Reporter: Dmytro Kulyk > Priority: Major > Attachments: Capture_after.JPG, Capture_before.JPG, Capture_error.JPG, Capture_tree.JPG, cube_update.py, cube_update_sit.py > > > I have a DAG created from function (see codescript and attachment below) > After uploading to server it become visible and can be executed by scheduler or manually > [^Capture_before.JPG] > [^Capture_tree.JPG] > However, in case of any activity upon this object (refresh etc) there is an "kaboom error" > [^Capture_error.JPG] > [^Capture_after.JPG] > {code} > ------------------------------------------------------------------------------- > Node: uat01.airflow.retail > ------------------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app > response = self.full_dispatch_request() > File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request > rv = self.handle_user_exception(e) > File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception > reraise(exc_type, exc_value, tb) > File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request > rv = self.dispatch_request() > File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 69, in inner > return self._run_view(f, *args, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 368, in _run_view > return fn(self, *args, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 755, in decorated_view > return func(*args, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 219, in view_func > return f(*args, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line 125, in wrapper > return f(*args, **kwargs) > File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line 1161, in tree > base_date = dag.latest_execution_date or datetime.now() > AttributeError: 'NoneType' object has no attribute 'latest_execution_date' > {code} > In case when DAG created directly (without function) - everything works as a charm > cube_update.py > {code} > from datetime import datetime > from lib.cube_update import cube_updater > env = 'sit' > start_date = datetime(2017, 7, 1) > schedule_interval = '0 8 * * *' > > dag = cube_updater(env, start_date, schedule_interval) > {code} > lib/__init__.py - empty one > lib/cube_update.py > {code} > from airflow import DAG > ... > def cube_updater(env, start_date, schedule_interval): > ... > dag = DAG( > dag_id = dag_id, > default_args = default_args, > description = 'OLAP Cube updater', > start_date = start_date, > schedule_interval = schedule_interval, > max_active_runs = 6, > catchup=True) > .... > return dag > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)