airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis Docter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1310) Kubernetes execute operator
Date Wed, 19 Jul 2017 17:40:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093479#comment-16093479
] 

Dennis Docter commented on AIRFLOW-1310:
----------------------------------------

It's similar to ours. We opted not to go with jobs, as kubernetes will then take care of scheduling,
retries etc, while that should be the responsibility of Airflow in this case.

Also you are not capturing any logs, preferably streaming, which would be a basic requirement
for the operator (especially since the pods get deleted afterwards)



> Kubernetes execute operator
> ---------------------------
>
>                 Key: AIRFLOW-1310
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1310
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: operators
>            Reporter: Yu Ishikawa
>            Assignee: Dennis Docter
>
> We should support operators for Kubernetes to execute containers remotely.
> - Create a Kubernetes cluster
> - Execute PODs
> - https://github.com/kubernetes-incubator/client-python



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message