airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-656) Lock wait timeout exceeded on XCOM table
Date Wed, 30 Nov 2016 18:43:58 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709383#comment-15709383
] 

ASF subversion and git services commented on AIRFLOW-656:
---------------------------------------------------------

Commit 05f7adb462f978e99a3c11ab276b96dc4a8d10b7 in incubator-airflow's branch refs/heads/master
from [~criccomini]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=05f7adb ]

[AIRFLOW-656] Add dag/task/date index to xcom table

Closes #1907 from criccomini/AIRFLOW-656


> Lock wait timeout exceeded on XCOM table
> ----------------------------------------
>
>                 Key: AIRFLOW-656
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-656
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: xcom
>    Affects Versions: Airflow 1.7.1.2
>            Reporter: Chris Riccomini
>            Assignee: Chris Riccomini
>
> We are seeing task failures with this periodically:
> {noformat}
> (_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout
> exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE
> xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND
> xcom.dag_id = %s'] [parameters: (u'return_value',
> datetime.datetime(2016, 11, 28, 15, 47),
> 'gcs2bq_fraud.fraud_rules_executions',
> 'db_monolith_fraud_fraud_rules_executions_partition_15m')]
> {noformat}
> According to [~maxime.beauchemin@apache.org]:
> {quote}
> I just looked quickly and it looks like no index is defined on the XCom
> model beyond the PK. We should add a composite btree index on dag_id,
> task_id and execution_date. I don't think MySQL will let you include `key`
> in there as it's too long (512). The 3 other fields are plenty selective so
> that will work just as well.
> Creating the index straight in the db should fix the problem right away in
> your environment, but it'd be nice to add it to the model definition and
> generate the db migration script for it.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message