airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rémy Frenoy (JIRA) <>
Subject [jira] [Created] (AIRFLOW-1583) PostgreSQL error when pushing a value bigger than 1GB
Date Fri, 08 Sep 2017 14:47:00 GMT
Rémy Frenoy created AIRFLOW-1583:

             Summary: PostgreSQL error when pushing a value bigger than 1GB
                 Key: AIRFLOW-1583
             Project: Apache Airflow
          Issue Type: Bug
          Components: xcom
    Affects Versions: Airflow 1.8, Airflow 1.7.0
            Reporter: Rémy Frenoy
            Assignee: Rémy Frenoy
            Priority: Minor

When a task Operator or a PythonOperator returns a value, this value is automatically pushed
to XCom. This results in a postgresql error when the value is bigger than 1GB :

ERROR:  invalid memory alloc request size 1073741824

A size limit could be added on the value size in the xcom-push() function in When
value is higher than the limit size, a default value (maybe a string explaining that the initial
value was too high to be saved) could be pushed to XCom instead of the real one, avoiding
the postgresql error.

This message was sent by Atlassian JIRA

View raw message