Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4212F200C63 for ; Thu, 11 May 2017 15:26:19 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 40B7D160BCA; Thu, 11 May 2017 13:26:19 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 427C6160BB2 for ; Thu, 11 May 2017 15:26:18 +0200 (CEST) Received: (qmail 22071 invoked by uid 500); 11 May 2017 13:26:17 -0000 Mailing-List: contact dev-help@ariatosca.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ariatosca.incubator.apache.org Delivered-To: mailing list dev@ariatosca.incubator.apache.org Received: (qmail 22054 invoked by uid 99); 11 May 2017 13:26:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 May 2017 13:26:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 195FB1A068C for ; Thu, 11 May 2017 13:26:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id A2kMOvHkssx1 for ; Thu, 11 May 2017 13:26:10 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id A303760FB5 for ; Thu, 11 May 2017 13:26:07 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E14C7E0DD2 for ; Thu, 11 May 2017 13:26:06 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6B71821E36 for ; Thu, 11 May 2017 13:26:05 +0000 (UTC) Date: Thu, 11 May 2017 13:26:05 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: dev@ariatosca.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (ARIA-213) Sporadic tests failures over locked database issue MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 11 May 2017 13:26:19 -0000 [ https://issues.apache.org/jira/browse/ARIA-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006412#comment-16006412 ] ASF GitHub Bot commented on ARIA-213: ------------------------------------- Github user mxmrlv commented on a diff in the pull request: https://github.com/apache/incubator-ariatosca/pull/129#discussion_r115976779 --- Diff: aria/storage/instrumentation.py --- @@ -53,30 +59,76 @@ def track_changes(instrumented=None): :param instrumented: A dict from model columns to their python native type :return: The instrumentation context """ - return _Instrumentation(instrumented or _INSTRUMENTED) + return _Instrumentation(ctx, instrumented or _INSTRUMENTED) class _Instrumentation(object): - def __init__(self, instrumented): + def __init__(self, ctx, instrumented): self.tracked_changes = {} + self.new_instances = {} self.listeners = [] + self._new_modeled_instances = [] + self._ctx = ctx self._track_changes(instrumented) + self._new_instance_index = 0 + + @property + def _new_instance_id(self): + self._new_instance_index += 1 + return '{prefix}_{index}'.format(prefix=_NEW_INSTANCE, index=self._new_instance_index - 1) --- End diff -- remove "-1" > Sporadic tests failures over locked database issue > -------------------------------------------------- > > Key: ARIA-213 > URL: https://issues.apache.org/jira/browse/ARIA-213 > Project: AriaTosca > Issue Type: Bug > Reporter: Ran Ziv > Assignee: Maxim Orlov > Priority: Minor > > Unit tests sporadically fail over an issue of locked database. > Here are two stack traces for the same issue that I found on a single run of all of the tests: > {code} > _________________ TestLocalRunScript.test_retry_with_interval __________________ > self = > executor = > workflow_context = WorkflowContext(deployment_id=1, workflow_name=test_workflow, execution_id=1) > tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_retry_with_interval0') > def test_retry_with_interval(self, executor, workflow_context, tmpdir): > script_path = self._create_script( > tmpdir, > linux_script='''#! /bin/bash -e > ctx task retry retry-message @100 > ''', > windows_script=''' > ctx task retry retry-message @100 > ''') > exception = self._run_and_get_task_exception( > executor, workflow_context, > > script_path=script_path) > tests/orchestrator/execution_plugin/test_local.py:323: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > tests/orchestrator/execution_plugin/test_local.py:452: in _run_and_get_task_exception > self._run(*args, **kwargs) > tests/orchestrator/execution_plugin/test_local.py:503: in _run > eng.execute() > aria/orchestrator/workflows/core/engine.py:60: in execute > for task in self._ended_tasks(): > aria/orchestrator/workflows/core/engine.py:97: in > return (task for task in self._tasks_iter() if task.has_ended()) > aria/orchestrator/workflows/core/engine.py:110: in _tasks_iter > self._workflow_context.model.task.refresh(task.model_task) > aria/orchestrator/workflows/core/task.py:191: in model_task > return self._workflow_context.model.task.get(self._task_id) > aria/storage/sql_mapi.py:59: in get > result = query.first() > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first > ret = list(self[0:1]) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in __getitem__ > return list(res) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in __iter__ > return self._execute_and_instances(context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in execute > return meth(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement > compiled_sql, distilled_params > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context > context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in _handle_dbapi_exception > exc_info > .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context > context) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > self = > cursor = > statement = 'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS task_implementation, task.max_attempts AS t...hip_fk AS task_relationship_fk, task.execution_fk AS task_execution_fk > FROM task > WHERE task.id = ? > LIMIT ? OFFSET ?' > parameters = (1, 1, 0) > context = > def do_execute(self, cursor, statement, parameters, context=None): > > cursor.execute(statement, parameters) > E OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT task.id AS task_id, task.name AS task_name, task.implementation AS task_implementation, task.max_attempts AS task_max_attempts, task.retry_interval AS task_retry_interval, task.ignore_failure AS task_ignore_failure, task.status AS task_status, task.due_at AS task_due_at, task.started_at AS task_started_at, task.ended_at AS task_ended_at, task.attempts_count AS task_attempts_count, task.plugin_fk AS task_plugin_fk, task.node_fk AS task_node_fk, task.relationship_fk AS task_relationship_fk, task.execution_fk AS task_execution_fk \nFROM task \nWHERE task.id = ?\n LIMIT ? OFFSET ?'] [parameters: (1, 1, 0)] > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: OperationalError > {code} > {code} > _______________ TestLocalRunScript.test_crash_abort_after_abort ________________ > self = > executor = > workflow_context = WorkflowContext(deployment_id=1, workflow_name=test_workflow, execution_id=1) > tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_crash_abort_after_abort0') > def test_crash_abort_after_abort(self, executor, workflow_context, tmpdir): > script_path = self._create_script( > tmpdir, > linux_script='''#! /bin/bash > ctx task abort abort-message > ctx task abort should-raise-a-runtime-error > ''', > windows_script=''' > ctx task abort abort-message > ctx task abort should-raise-a-runtime-error > ''') > exception = self._run_and_get_task_exception( > executor, workflow_context, > > script_path=script_path) > tests/orchestrator/execution_plugin/test_local.py:375: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > tests/orchestrator/execution_plugin/test_local.py:452: in _run_and_get_task_exception > self._run(*args, **kwargs) > tests/orchestrator/execution_plugin/test_local.py:503: in _run > eng.execute() > aria/orchestrator/workflows/core/engine.py:57: in execute > cancel = self._is_cancel() > aria/orchestrator/workflows/core/engine.py:86: in _is_cancel > return self._workflow_context.execution.status in (models.Execution.CANCELLING, > aria/orchestrator/context/workflow.py:61: in execution > return self.model.execution.get(self._execution_id) > aria/storage/sql_mapi.py:59: in get > result = query.first() > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2731: in first > ret = list(self[0:1]) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2523: in __getitem__ > return list(res) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2831: in __iter__ > return self._execute_and_instances(context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/orm/query.py:2854: in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:945: in execute > return meth(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement > compiled_sql, distilled_params > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context > context) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1393: in _handle_dbapi_exception > exc_info > .tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context > context) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > self = > cursor = > statement = 'SELECT execution.id AS execution_id, execution.name AS execution_name, execution.created_at AS execution_created_at, ..._workflow_name, execution.service_fk AS execution_service_fk > FROM execution > WHERE execution.id = ? > LIMIT ? OFFSET ?' > parameters = (1, 1, 0) > context = > def do_execute(self, cursor, statement, parameters, context=None): > > cursor.execute(statement, parameters) > E OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT execution.id AS execution_id, execution.name AS execution_name, execution.created_at AS execution_created_at, execution.started_at AS execution_started_at, execution.ended_at AS execution_ended_at, execution.error AS execution_error, execution.status AS execution_status, execution.workflow_name AS execution_workflow_name, execution.service_fk AS execution_service_fk \nFROM execution \nWHERE execution.id = ?\n LIMIT ? OFFSET ?'] [parameters: (1, 1, 0)] > .tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: OperationalError > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)