From dev-return-7942-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Sun Mar 31 02:57:30 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9B342180626 for ; Sun, 31 Mar 2019 04:57:29 +0200 (CEST) Received: (qmail 53449 invoked by uid 500); 31 Mar 2019 02:57:28 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 53437 invoked by uid 99); 31 Mar 2019 02:57:27 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 31 Mar 2019 02:57:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 115ECC7377 for ; Sun, 31 Mar 2019 02:57:27 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.049 X-Spam-Level: ** X-Spam-Status: No, score=2.049 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id zDJJkKpw3WHs for ; Sun, 31 Mar 2019 02:57:26 +0000 (UTC) Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id E39F65FE57 for ; Sun, 31 Mar 2019 02:57:25 +0000 (UTC) Received: by mail-yb1-f176.google.com with SMTP id a123so541795ybg.10 for ; Sat, 30 Mar 2019 19:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=9f6ernp/p6tsqi20de/cIvJdO4m+b3F+BJDns1YDRi8=; b=CcKrAJ2omftoKXxzbTILIkYoQtZjt6Q/sdAy6p3YSLTERumdyZb3FaqFTN8w2xozUU ARfrxALbWC7aoNbRghiSb4ayh+YQKW7rIaCuXfIrqY+xf73tqSfh8gLFRSy9BT3SS4TI f8FdCCvNvwHfHLE6FrCeP92lWUHHeqVJ2MjfopL5N+bJJ4y5UfDKBf+exVHkDCoG3Z90 pXmj18IkERLiCXstXPyiuo6tAPJb6cHkAECs53BYkiN/se/Yr6LHWvCQG42sa5jHysvK U8MKFiN/mQGqFzRmeYhEVp6rjiJ/MchXBT9VifmULHXFMXKvtGvK9Hev1eKEdil6YGwW opRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=9f6ernp/p6tsqi20de/cIvJdO4m+b3F+BJDns1YDRi8=; b=AQ35bhs/cTqHbVNQERcLkhK1aIaoL2rTfyIrtZXSMVcrMDRHRKDkBuPZDZUvGVBkBi zIEgZ6pj03uXZlKEz7QjKcX1qcMKFlidRG8W5qCFgfpak/fpPsUGVF4sxeSO1t3tMsTk /67ji/EOw8/YeMlbBF9flIrykm0qGHUlbV7PYKyDHz/kEhnK/hqS1Se7V81wrUKeXjSJ lbwA/ouVtKvnDwlyA7aBNyFdFecM3XH0KV7szVNJEDTiaM7+7cnETVyjQ80rrDb5hGUS h7oYeRnefTUvhRr/BsdxWFAKpJgpgkzw06/4SPzacHiopP0OX6OBfbqjUnrHtHRjPQuC NEQA== X-Gm-Message-State: APjAAAX6wX3WTHLQAicZZWdWl3eQWTxg1Gs12X/e0nRdFTt9OVoyND0c X25JyR0DaVBtdIK9WRZNm7aw5kezZFr2X5T/An+CcA== X-Google-Smtp-Source: APXvYqx3K8WL19/lrck2YR9uedd+7C3R9I1ErW8Xl/4tyg5ksE4q/M/ik1D6TYzDDrPnfPghFOoh7DrzJETsQgdcowU= X-Received: by 2002:a25:c04a:: with SMTP id c71mr1075020ybf.470.1554001039365; Sat, 30 Mar 2019 19:57:19 -0700 (PDT) MIME-Version: 1.0 From: Chao-Han Tsai Date: Sat, 30 Mar 2019 19:57:08 -0700 Message-ID: Subject: [DISCUSS] Utility DAGs To: dev@airflow.apache.org Content-Type: multipart/alternative; boundary="000000000000ee9b6c05855b0f7f" --000000000000ee9b6c05855b0f7f Content-Type: text/plain; charset="UTF-8" Hi all, I have been thinking about adding some DAGs that are for the purpose of AIrflow cluster operation, DAG schedule delay instrumentation and log retention for instance. Currently we have example_dags, should we add another directory utility_dags in the repo? We can have a flag in airflow.cfg to let user decide whether to load the utility_dags (just like what we did for example_dags). What do you think? -- Chao-Han Tsai --000000000000ee9b6c05855b0f7f--