Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9CC3D200CB3 for ; Mon, 26 Jun 2017 12:28:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9B3DA160BDE; Mon, 26 Jun 2017 10:28:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id BAB98160BD9 for ; Mon, 26 Jun 2017 12:28:05 +0200 (CEST) Received: (qmail 3332 invoked by uid 500); 26 Jun 2017 10:28:04 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 3322 invoked by uid 99); 26 Jun 2017 10:28:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Jun 2017 10:28:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 8B7CBC061B for ; Mon, 26 Jun 2017 10:28:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id EOJPUDsxJg9t for ; Mon, 26 Jun 2017 10:28:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id AF46A5FDCC for ; Mon, 26 Jun 2017 10:28:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 0E33CE069F for ; Mon, 26 Jun 2017 10:28:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 40F8E240CD for ; Mon, 26 Jun 2017 10:28:00 +0000 (UTC) Date: Mon, 26 Jun 2017 10:28:00 +0000 (UTC) From: "Antoine Grouazel (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AIRFLOW-1347) sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL for primary key MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 26 Jun 2017 10:28:06 -0000 [ https://issues.apache.org/jira/browse/AIRFLOW-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Grouazel updated AIRFLOW-1347: -------------------------------------- Attachment: bug_airflow.txt > sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL for primary key > ------------------------------------------------------------------------------------------ > > Key: AIRFLOW-1347 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1347 > Project: Apache Airflow > Issue Type: Bug > Components: DagRun, db > Affects Versions: 1.8.0 > Environment: ubuntu 12.04 > mysql 5.7.18 > Reporter: Antoine Grouazel > Attachments: bug_airflow.txt > > > Description of the issue: the scheduler cannot update the mysql db via SqlAlchemy and the tasks are queued and not rescheduled. It affects all our DAGs but not all the tasks nor all the time. Manually clear a task having such issue can fix the problem for the given task instance. > There is a kind of random behavior... > Upgrading mysql 5.7.17 to 5.7.18 does not fix the issue. > Issue is neitheir fixed in the lastest version 1.8.2rc1 . > Don't know if there is a link with https://issues.apache.org/jira/browse/AIRFLOW-977. > {{[2017-06-24 00:31:10,272] {models.py:167} INFO - Filling up the DagBag from /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py > [2017-06-24 00:31:10,427] {base_task_runner.py:112} INFO - Running: ['bash', '-c', u'airflow run Sentinel1_copernicus_download latest_only 2017-06-23T00:31:00 --job_id 293 --raw -sd DAGS_FOLDER/sentinel1_copernicus_download.py'] > [2017-06-24 00:31:11,449] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,449] {__init__.py:57} INFO - Using executor CeleryExecutor > [2017-06-24 00:31:11,668] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,668] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt > [2017-06-24 00:31:11,724] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,723] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt > [2017-06-24 00:31:12,067] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,066] {models.py:167} INFO - Filling up the DagBag from /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py > [2017-06-24 00:31:12,184] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,184] {models.py:1126} INFO - Dependencies all met for > [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,188] {models.py:1126} INFO - Dependencies all met for > [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,188] {models.py:1318} INFO - > [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: -------------------------------------------------------------------------------- > [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: Starting attempt 1 of 3 > [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask: -------------------------------------------------------------------------------- > [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask: > [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last): > [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/bin/airflow", line 28, in > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: args.func(args) > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 422, in run > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: pool=args.pool, > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 53, in wrapper > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: result = func(*args, **kwargs) > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1328, in run > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: session.commit() > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 801, in commit > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: self.transaction.commit() > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 392, in commit > [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: self._prepare_impl() > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 372, in _prepare_impl > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: self.session.flush() > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: self._flush(objects) > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: transaction.rollback(_capture_exception=True) > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__ > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: compat.reraise(exc_type, exc_value, exc_tb) > [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: flush_context.execute() > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: rec.execute(self) > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: uow > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 170, in save_obj > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: mapper, table, update) > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 634, in _emit_update_statements > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: lambda rec: ( > [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 531, in _collect_update_commands > [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask: "key value on column %s" % (table, col)) > [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask: sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL for primary key value on column task_instance.execution_date > [2017-06-24 00:31:12,311] {jobs.py:2083} INFO - Task exited with return code 1}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)