From dev-return-9159-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Tue Aug 13 18:55:12 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 619711804BB for ; Tue, 13 Aug 2019 20:55:12 +0200 (CEST) Received: (qmail 14542 invoked by uid 500); 13 Aug 2019 18:55:11 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 14530 invoked by uid 99); 13 Aug 2019 18:55:11 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2019 18:55:11 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 68847C07A7 for ; Tue, 13 Aug 2019 18:55:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 0Q9KXnV8L4Si for ; Tue, 13 Aug 2019 18:55:08 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=199.53.195.150; helo=mail-gw31.credit-suisse.com; envelope-from=damian.shaw.2@credit-suisse.com; receiver= Received: from mail-gw31.credit-suisse.com (mail-gw31.credit-suisse.com [199.53.195.150]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 40C35BC7A9 for ; Tue, 13 Aug 2019 18:55:07 +0000 (UTC) Received: by mail-gw31.credit-suisse.com; Tue, 13 Aug 2019 18:54:58 GMT From: "Shaw, Damian P. " To: "dev@airflow.apache.org" , Chris Palmer Subject: RE: Manipulating the DAG Code View? Thread-Topic: Manipulating the DAG Code View? Thread-Index: AdVSAz6DOtpVM8vFTSanT6v8iMalWQAJaLSAAAALLQAACEHXIA== Date: Tue, 13 Aug 2019 18:54:53 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-sentfromoutlook: SentFromOutlook Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit MIME-Version: 1.0 As Ash says, I tried creating some test DAGs in 1.10.x and found that they didn't run. There wasn't even any obvious error, Airflow loaded the file (I could see pyc created) but it never appeared in the DAG bag. Damian -----Original Message----- From: Ash Berlin-Taylor [mailto:ash@apache.org] Sent: Tuesday, August 13, 2019 2:47 PM To: dev@airflow.apache.org; Chris Palmer Subject: Re: Manipulating the DAG Code View? Be carefully doing that in 1.10.4 - if you change that you may find your dags don't run anymore! On 13 August 2019 19:46:01 BST, Chris Palmer wrote: >A more involved PR like you suggest might be valuable in the long run, >but >in the short term I've been successful in the past with simply >modifying >the fileloc attribute of DAGs. It get's set here > >to the previous frame, but you could override it once you've created >the >DAG object. > >Chris > >On Tue, Aug 13, 2019 at 2:31 PM Shaw, Damian P. < >damian.shaw.2@credit-suisse.com> wrote: > >> Hi all, >> >> I'm in need of being able to manipulate the code view for many of my >DAGs >> which are generated by a single Python script that reads config >files. My >> DAG creation script is getting increasingly less useful to the end >users so >> I would like to instead present the config file that was used to >generate >> the DAG. >> >> I took a look at the relevant code vide code in Airflow: >> >https://github.com/apache/airflow/blob/3e2a02751cf890b780bc26b40c7cee7f1f4e0bd9/airflow/www/views.py#L433 >> >> It seems it would be a relatively easy change to add an extra option >on >> the DAG, e.g. "dag.code_display_file" and maybe also a Lexer option, >then >> detect if this option has been set for the DAG in the above location >and >> display the specified file instead. >> >> I don't have any experience contributing to open source projects of >this >> size, would this be a welcome pull request? I'd rather not waste >anyone's >> time if it isn't. >> >> Thanks, >> Damian >> >> >=============================================================================== >> >> Please access the attached hyperlink for an important electronic >> communications disclaimer: >> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >> >=============================================================================== >> >> =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===============================================================================