Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 43585200CA4 for ; Wed, 7 Jun 2017 16:55:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 41FA1160BD0; Wed, 7 Jun 2017 14:55:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 39FC0160BB6 for ; Wed, 7 Jun 2017 16:55:24 +0200 (CEST) Received: (qmail 19308 invoked by uid 500); 7 Jun 2017 14:55:23 -0000 Mailing-List: contact issues-help@ariatosca.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ariatosca.incubator.apache.org Delivered-To: mailing list issues@ariatosca.incubator.apache.org Received: (qmail 19299 invoked by uid 99); 7 Jun 2017 14:55:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Jun 2017 14:55:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 141B01AFEBA for ; Wed, 7 Jun 2017 14:55:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id iKIwxBHfcDOh for ; Wed, 7 Jun 2017 14:55:20 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id D26A95FC84 for ; Wed, 7 Jun 2017 14:55:19 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E333AE0D95 for ; Wed, 7 Jun 2017 14:55:18 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3E1A921E10 for ; Wed, 7 Jun 2017 14:55:18 +0000 (UTC) Date: Wed, 7 Jun 2017 14:55:18 +0000 (UTC) From: "Maxim Orlov (JIRA)" To: issues@ariatosca.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (ARIA-213) Sporadic tests failures over locked database issue MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 07 Jun 2017 14:55:25 -0000 [ https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040948#comment-16040948 ] Maxim Orlov edited comment on ARIA-213 at 6/7/17 2:55 PM: ---------------------------------------------------------- database locked is a know issue when working with sqlalcehmy as pysqlite as a driver. pysqlite tries to "guess" where to start and end transactions. pysqlite sometimes fails in this prediction which might result in database_locked issue. Overriding this behavior is possible via sqlalchemy. However, this raises other issues. (ref: http://docs.sqlalchemy.org/en/latest/dialects/sqlite.html#serializable-isolation-savepoints-transactional-ddl) Another way to try and solve this locking issue is to increase the timeout for acquiring locks. Thus, we increased the timeout from the default (5 secs) to 15. was (Author: mxmrlv): database is a know issue when working with sqlalcehmy as pysqlite as a driver. pysqlite tries to "guess" where to start and end transactions. pysqlite sometimes fails in this prediction which might result in database_locked issue. Overriding this behavior is possible via sqlalchemy. However, this raises other issues. (ref: http://docs.sqlalchemy.org/en/latest/dialects/sqlite.html#serializable-isolation-savepoints-transactional-ddl) Another way to try and solve this locking issue is to increase the timeout for acquiring locks. Thus, we increased the timeout from the default (5 secs) to 15. > Sporadic tests failures over locked database issue > -------------------------------------------------- > > Key: ARIA-213 > URL: https://issues.apache.org/jira/browse/ARIA-213 > Project: AriaTosca > Issue Type: Bug > Reporter: Ran Ziv > Assignee: Maxim Orlov > Priority: Minor > Fix For: 0.1.0 > > > Unit tests sporadically fail over an issue of locked database. > Here are two stack traces for the same issue that I found on a single run of all of the tests: > {code} > _________________ TestLocalRunScript.test_retry_with_interval __________________ > self = > executor = > workflow_context = WorkflowContext(deployment_id=1, workflow_name=test_workflow, execution_id=1) > tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0') > def test_retry_with_interval(self, executor, workflow_context, tmpdir): > script_path = self._create_script( > tmpdir, > linux_script='''#! /bin/bash -e > ctx task retry retry-message @100 > ''', > windows_script=''' > ctx task retry retry-message @100 > ''') > exception = self._run_and_get_task_exception( > executor, workflow_context, > > script_path=script_path) > tests/orchestrator/execution_plugin/test_local.py:323: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > tests/orchestrator/execution_plugin/test_local.py:452: in _run_and_get_task_exception > self._run(*args, **kwargs) > tests/orchestrator/execution_plugin/test_local.py:503: in _run > eng.execute() > aria/orchestrator/workflows/core/engine.py:60: in execute > for task in self._ended_tasks(): > aria/orchestrator/workflows/core/engine.py:97: in > return (task for task in self._tasks_iter() if task.has_ended()) > aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter > self._workflow_context.model.task.refresh(task.model_task) > aria/orchestrator/workflows/core/task.py:191: in model_task > return self._workflow_context.model.task.get(self._task_id) > aria/storage/sql_mapi.py:59: in get > result = query.first() > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first > ret = list(self[0:1]) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in __getitem__ > return list(res) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in __iter__ > return self._execute_and_instances(context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in execute > return meth(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement > compiled_sql, distilled_params > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context > context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in _handle_dbapi_exception > exc_info > .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context > context) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > self = > cursor = > statement = 'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS task_implementation, task.max_attempts AS t...hip_fk AS task_relationship_fk, task.execution_fk AS task_execution_fk > FROM task > WHERE task.id = ? > LIMIT ? OFFSET ?' > parameters = (1, 1, 0) > context = > def do_execute(self, cursor, statement, parameters, context=None): > > cursor.execute(statement, parameters) > E OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS task_implementation, task.max_attempts AS task_max_attempts, task.retry_interval AS task_retry_interval, task.ignore_failure AS task_ignore_failure, task.status AS task_status, task.due_at AS task_due_at, task.started_at AS task_started_at, task.ended_at AS task_ended_at, task.attempts_count AS task_attempts_count, task.plugin_fk AS task_plugin_fk, task.node_fk AS task_node_fk, task.relationship_fk AS task_relationship_fk, task.execution_fk AS task_execution_fk \nFROM task \nWHERE task.id = ?\n LIMIT ? OFFSET ?'] [parameters: (1, 1, 0)] > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: OperationalError > {code} > {code} > _______________ TestLocalRunScript.test_crash_abort_after_abort ________________ > self = > executor = > workflow_context = WorkflowContext(deployment_id=1, workflow_name=test_workflow, execution_id=1) > tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_crash_abort_after_abort0') > def test_crash_abort_after_abort(self, executor, workflow_context, tmpdir): > script_path = self._create_script( > tmpdir, > linux_script='''#! /bin/bash > ctx task abort abort-message > ctx task abort should-raise-a-runtime-error > ''', > windows_script=''' > ctx task abort abort-message > ctx task abort should-raise-a-runtime-error > ''') > exception = self._run_and_get_task_exception( > executor, workflow_context, > > script_path=script_path) > tests/orchestrator/execution_plugin/test_local.py:375: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > tests/orchestrator/execution_plugin/test_local.py:452: in _run_and_get_task_exception > self._run(*args, **kwargs) > tests/orchestrator/execution_plugin/test_local.py:503: in _run > eng.execute() > aria/orchestrator/workflows/core/engine.py:57: in execute > cancel = self._is_cancel() > aria/orchestrator/workflows/core/engine.py:86: in _is_cancel > return self._workflow_context.execution.status in (models.Execution.CANCELLING, > aria/orchestrator/context/workflow.py:61: in execution > return self.model.execution.get(self._execution_id) > aria/storage/sql_mapi.py:59: in get > result = query.first() > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first > ret = list(self[0:1]) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in __getitem__ > return list(res) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in __iter__ > return self._execute_and_instances(context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in execute > return meth(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement > compiled_sql, distilled_params > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context > context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in _handle_dbapi_exception > exc_info > .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context > context) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > self = > cursor = > statement = 'SELECT execution.id AS execution_id, execution.name AS execution_name, execution.created_at AS execution_created_at, ..._workflow_name, execution.service_fk AS execution_service_fk > FROM execution > WHERE execution.id = ? > LIMIT ? OFFSET ?' > parameters = (1, 1, 0) > context = > def do_execute(self, cursor, statement, parameters, context=None): > > cursor.execute(statement, parameters) > E OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT execution.id AS execution_id, execution.name AS execution_name, execution.created_at AS execution_created_at, execution.started_at AS execution_started_at, execution.ended_at AS execution_ended_at, execution.error AS execution_error, execution.status AS execution_status, execution.workflow_name AS execution_workflow_name, execution.service_fk AS execution_service_fk \nFROM execution \nWHERE execution.id = ?\n LIMIT ? OFFSET ?'] [parameters: (1, 1, 0)] > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: OperationalError > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)