airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "e b (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-2313) Add ClusterLifecycleConfig to DataprocClusterCreateOperator
Date Wed, 11 Apr 2018 07:48:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

e b updated AIRFLOW-2313:
-------------------------
    Description: 
Google introduced [Cloud Dataproc Cluster Scheduled Deletion|[https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scheduled-deletion]]
feature which isn't yet supported by the DataprocClusterCreateOperator.

 

Having this option enabled could be useful to make sure Dataproc cluster doesn't stay active
for extended time periods (In case delete DataprocClusterDeleteOperator isn't suitable or
fails to run for any reason).

 

Implementation requires adding config object [LifecycleConfig|https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/ClusterConfig#LifecycleConfig]

 

 

  was:
Google introduced [Cloud Dataproc Cluster Scheduled Deletion (TTL)|[https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scheduled-deletion]] feature which
isn't yet supported by the DataprocClusterCreateOperator.

 

Having this option enabled could be useful to make sure Dataproc cluster doesn't stay active
for extended time periods (In case delete DataprocClusterDeleteOperator isn't suitable or
fails to run for any reason).

 

Implementation requires adding config object [LifecycleConfig|https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/ClusterConfig#LifecycleConfig]

 

 


> Add ClusterLifecycleConfig to DataprocClusterCreateOperator
> -----------------------------------------------------------
>
>                 Key: AIRFLOW-2313
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2313
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, gcp, operators
>    Affects Versions: Airflow 2.0, 1.9.0
>            Reporter: e b
>            Priority: Minor
>
> Google introduced [Cloud Dataproc Cluster Scheduled Deletion|[https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scheduled-deletion]]
feature which isn't yet supported by the DataprocClusterCreateOperator.
>  
> Having this option enabled could be useful to make sure Dataproc cluster doesn't stay
active for extended time periods (In case delete DataprocClusterDeleteOperator isn't suitable
or fails to run for any reason).
>  
> Implementation requires adding config object [LifecycleConfig|https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/ClusterConfig#LifecycleConfig]
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message