From commits-return-86024-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Fri Jan 3 14:11:39 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id B9DB318061A for ; Fri, 3 Jan 2020 15:11:38 +0100 (CET) Received: (qmail 10762 invoked by uid 500); 3 Jan 2020 14:11:38 -0000 Mailing-List: contact commits-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list commits@airflow.apache.org Received: (qmail 10753 invoked by uid 99); 3 Jan 2020 14:11:38 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Jan 2020 14:11:38 +0000 From: GitBox To: commits@airflow.apache.org Subject: [GitHub] [airflow-site] kaxil commented on a change in pull request #231: Add blogs for airflow DAGs testing Message-ID: <157806069794.8155.18196781539038408679.gitbox@gitbox.apache.org> References: In-Reply-To: Date: Fri, 03 Jan 2020 14:11:37 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit kaxil commented on a change in pull request #231: Add blogs for airflow DAGs testing URL: https://github.com/apache/airflow-site/pull/231#discussion_r362821134 ########## File path: landing-pages/site/content/en/blog/testing-in-airflow-part-1-dag-validation-tests-dag-definition-tests-and-unit-tests.md ########## @@ -0,0 +1,16 @@ +--- +title: "Testing in Airflow Part 1 - DAG Validation Tests DAG, Definition Tests and Unit Tests" +linkTitle: "Testing in Airflow Part 1 - DAG Validation Tests, DAG Definition Tests and Unit Tests" +author: "Chandu Kavar" +twitter: "chandukavar" +github: "chandulal" +linkedin: "chandu-kavar-627a209b" +description: "To know how to test the validity of DAGs, typos, cyclicity, upstream and downstream dependencies and custom operator/sensors." +tags: ["Testing"] +date: "2020-01-03" +--- +Testing is an integral part of any software system to build confidence and increase the reliability of the system. Recently, I joined Grab and here at Grab, we are using Airflow to create and manage pipelines. But, we were facing issues with Airflow. I had a conversation with my engineering manager and discussed on how we could make Airflow reliable and testable. +Before Grab, I worked with ThoughtWorks. They are using TDD (Test Driven Development) methodology in almost all the projects. As a matter of fact, everyone is very cognizant about testing. This motivated me to explore testing in Airflow. +I spent a weekend Googling. But, I didn’t find any good articles on airflow test covering all the aspects, there are only a handful of articles available. I felt it’s worth to write and share a blog on airflow testing. + +You can find more details on how to write these tests with examples in [Testing in Airflow Part 1](https://blog.usejournal.com/testing-in-airflow-part-1-dag-validation-tests-dag-definition-tests-and-unit-tests-2aa94970570c?source=friends_link&sk=3e1ec0b1e5fcc1e9c258e08d260c7152). Review comment: Sorry, the content needs to entirely on the Airflow website. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services