airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <>
Subject [jira] [Commented] (AIRFLOW-3783) Switch to HEADER arg to unload data from Redshift to S3
Date Mon, 14 Oct 2019 11:42:01 GMT


ASF subversion and git services commented on AIRFLOW-3783:

Commit 5447b153665cc48b857f99545898172f76b734da in airflow's branch refs/heads/v1-10-test
from Felix Uellendall
[;h=5447b15 ]

[AIRFLOW-3783] Speed up Redshift to S3 UNload with HEADERs (#6309)

replace the default unload_option 'PARALLEL OFF' by 'HEADER' as Redshift handles
header with parallel mode now

Co-authored-by: Shreya Chakraborty <>
(cherry picked from commit 3ed7b32775cf8328836cfb36af8e65cab01e221b)

> Switch to HEADER arg to unload data from Redshift to S3
> -------------------------------------------------------
>                 Key: AIRFLOW-3783
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: operators
>    Affects Versions: 1.10.5
>            Reporter: Shreya Chakraborty
>            Assignee: Felix Uellendall
>            Priority: Major
>             Fix For: 1.10.6
> Redshift has added a new *HEADER* option to the s3unload parameters.
> So we can remove the default PARALLEL OFF in the existing
> RedshiftToS3Transfer operator. The unload operation can now run parallely, hence faster,
along with headers, if required.

This message was sent by Atlassian Jira

View raw message