From commits-return-20756-archive-asf-public=cust-asf.ponee.io@airflow.incubator.apache.org Tue Sep 4 00:54:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 25E49180647 for ; Tue, 4 Sep 2018 00:54:03 +0200 (CEST) Received: (qmail 44402 invoked by uid 500); 3 Sep 2018 22:54:03 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 44393 invoked by uid 99); 3 Sep 2018 22:54:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Sep 2018 22:54:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id D28EEC0558 for ; Mon, 3 Sep 2018 22:54:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id EUpBZ3s6XgsK for ; Mon, 3 Sep 2018 22:54:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 3CB2D5F381 for ; Mon, 3 Sep 2018 22:54:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6AB24E0047 for ; Mon, 3 Sep 2018 22:54:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1C8A723F98 for ; Mon, 3 Sep 2018 22:54:00 +0000 (UTC) Date: Mon, 3 Sep 2018 22:54:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (AIRFLOW-251) Add optional parameter SQL_ALCHEMY_SCHEMA to control schema for metadata repository MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/AIRFLOW-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602477#comment-16602477 ] ASF GitHub Bot commented on AIRFLOW-251: ---------------------------------------- kaxil closed pull request #1600: [AIRFLOW-251] Add option SQL_ALCHEMY_SCHEMA parameter to use SQL Server for metadata. URL: https://github.com/apache/incubator-airflow/pull/1600 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/airflow/configuration.py b/airflow/configuration.py index e03b713046..03f15d721d 100644 --- a/airflow/configuration.py +++ b/airflow/configuration.py @@ -106,6 +106,7 @@ def run_command(command): 'dags_are_paused_at_creation': True, 'sql_alchemy_pool_size': 5, 'sql_alchemy_pool_recycle': 3600, + 'sql_alchemy_schema': None, 'dagbag_import_timeout': 30, 'non_pooled_task_slot_count': 128, }, @@ -210,6 +211,10 @@ def run_command(command): # their website sql_alchemy_conn = sqlite:///{AIRFLOW_HOME}/airflow.db +# The schema to use for the metadata database +# SqlAlchemy supports databases with the concept of multiple schemas. +sql_alchemy_schema = + # The SqlAlchemy pool size is the maximum number of database connections # in the pool. sql_alchemy_pool_size = 5 diff --git a/airflow/models.py b/airflow/models.py index 1b613188ca..cf8adfa04f 100644 --- a/airflow/models.py +++ b/airflow/models.py @@ -47,7 +47,7 @@ from sqlalchemy import ( Column, Integer, String, DateTime, Text, Boolean, ForeignKey, PickleType, Index, Float) -from sqlalchemy import case, func, or_, and_ +from sqlalchemy import case, func, or_, and_, MetaData from sqlalchemy.ext.declarative import declarative_base, declared_attr from sqlalchemy.dialects.mysql import LONGTEXT from sqlalchemy.orm import relationship, synonym @@ -71,7 +71,11 @@ from airflow.utils.timeout import timeout from airflow.utils.trigger_rule import TriggerRule -Base = declarative_base() +SQL_ALCHEMY_SCHEMA = configuration.get('core', 'SQL_ALCHEMY_SCHEMA') +if SQL_ALCHEMY_SCHEMA is None or SQL_ALCHEMY_SCHEMA.isspace(): + Base = declarative_base() +else: + Base = declarative_base(metadata=MetaData(schema=SQL_ALCHEMY_SCHEMA)) ID_LEN = 250 SQL_ALCHEMY_CONN = configuration.get('core', 'SQL_ALCHEMY_CONN') DAGS_FOLDER = os.path.expanduser(configuration.get('core', 'DAGS_FOLDER')) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org > Add optional parameter SQL_ALCHEMY_SCHEMA to control schema for metadata repository > ----------------------------------------------------------------------------------- > > Key: AIRFLOW-251 > URL: https://issues.apache.org/jira/browse/AIRFLOW-251 > Project: Apache Airflow > Issue Type: Improvement > Components: core > Affects Versions: 2.0.0 > Reporter: Ed Parcell > Priority: Minor > > Using SQL Server as a database for metadata, it is preferable to group all Airflow tables into a separate schema, rather than using dbo. I propose adding an optional parameter SQL_ALCHEMY_SCHEMA to control this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)