From dev-return-7185-archive-asf-public=cust-asf.ponee.io@airflow.incubator.apache.org Mon Dec 10 16:12:57 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 210EF180627 for ; Mon, 10 Dec 2018 16:12:56 +0100 (CET) Received: (qmail 31581 invoked by uid 500); 10 Dec 2018 15:12:56 -0000 Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list dev@airflow.incubator.apache.org Received: (qmail 31570 invoked by uid 99); 10 Dec 2018 15:12:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Dec 2018 15:12:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 912BFCF2E0 for ; Mon, 10 Dec 2018 15:12:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.972 X-Spam-Level: X-Spam-Status: No, score=0.972 tagged_above=-999 required=6.31 tests=[SPF_SOFTFAIL=0.972] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id MukSn9wAmkKC for ; Mon, 10 Dec 2018 15:12:53 +0000 (UTC) Received: from mail.firemirror.com (mail.firemirror.com [178.63.242.223]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 325D460F82 for ; Mon, 10 Dec 2018 15:12:53 +0000 (UTC) Received: from [217.169.25.231] (helo=themisto.localdomain) by mail.firemirror.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.90_1) (envelope-from ) id 1gWNEg-00033n-07 for dev@airflow.incubator.apache.org; Mon, 10 Dec 2018 15:12:47 +0000 From: Ash Berlin-Taylor Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Recommended backend metastore for Airflow Date: Mon, 10 Dec 2018 15:12:41 +0000 References: To: dev@airflow.incubator.apache.org In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3273) Postgres. Friends don't let friends use MySQL is my personal rule. (I can get in to the reasons if you'd like, but the short version is I = find Postgres has more compliant behaviour with SQL standard, and a much = better query planner.) -ash > On 10 Dec 2018, at 15:10, ramandumcs@gmail.com wrote: >=20 > Hi All, >=20 > It seems that Airflow supports mysql, postgresql and mssql as backend = store. Any recommendation on using one over other. We are expecting to = run 1000(s) of concurrent Dags which would generate heavy load on = backend store. > Any pointer on this would be useful. >=20 > Thanks, > Raman Gupta