airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2797) Add ability to create Google Dataproc cluster with custom image
Date Sun, 02 Sep 2018 17:59:03 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601330#comment-16601330
] 

Apache Spark commented on AIRFLOW-2797:
---------------------------------------

User 'ashb' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/3602

> Add ability to create Google Dataproc cluster with custom image
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-2797
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2797
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, gcp, operators
>    Affects Versions: 2.0.0, 1.10
>            Reporter: Jarosław Śmietanka
>            Assignee: Jarosław Śmietanka
>            Priority: Minor
>
> In GCP, it is possible to create Dataproc cluster with a [custom image|https://cloud.google.com/dataproc/docs/guides/dataproc-images]
that includes user's pre-installed packages. It significantly reduces the startup time of
the cluster.
>  
> Since I already have a code which does that, I volunteer to bring it to the community :)  
> This improvement won't change many components and should not require groundbreaking
changes to DataprocClusterCreateOperator.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message