airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fokko Driesprong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-2313) Add ClusterLifecycleConfig to DataprocClusterCreateOperator
Date Thu, 03 May 2018 21:14:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fokko Driesprong resolved AIRFLOW-2313.
---------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

> Add ClusterLifecycleConfig to DataprocClusterCreateOperator
> -----------------------------------------------------------
>
>                 Key: AIRFLOW-2313
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2313
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, gcp, operators
>    Affects Versions: Airflow 2.0, 1.9.0
>            Reporter: Egidijus Bartkus
>            Assignee: Egidijus Bartkus
>            Priority: Minor
>             Fix For: 2.0.0
>
>
> Google introduced [Cloud Dataproc Cluster Scheduled Deletion|https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scheduled-deletion]
feature which isn't yet supported by the DataprocClusterCreateOperator.
>  
> Having this option enabled could be useful to make sure Dataproc cluster doesn't stay
active for extended time periods (In case delete DataprocClusterDeleteOperator isn't suitable
or fails to run for any reason).
>  
> Implementation requires adding config object [LifecycleConfig|https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/ClusterConfig#LifecycleConfig]
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message