airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michal Dziemianko (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-1310) Kubernetes execute operator
Date Thu, 20 Jul 2017 08:10:00 GMT


Michal Dziemianko commented on AIRFLOW-1310:

It was exactly the same sort of thinking on my side. 

Good point about the logs. It was sort of design decision as we capture all logs centrally
in ELK so we had no need to stream the logs to airflow as such. It is not hard to add though.

Also regarding your comment that it would be more powerful if you can supply a pod definition
directly - thats another decision to hide the pod definition as such to make deployment of
simple jobs simpler. I have a version where you can give a pod definition instead in case
you need to do something complex, but for most jobs we have we just have an image (often with
default command) and we dont want to force people to learn yet another library. 

Do you mind collaborating to get out something nice and useful to everybody?

> Kubernetes execute operator
> ---------------------------
>                 Key: AIRFLOW-1310
>                 URL:
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: operators
>            Reporter: Yu Ishikawa
>            Assignee: Dennis Docter
> We should support operators for Kubernetes to execute containers remotely.
> - Create a Kubernetes cluster
> - Execute PODs
> -

This message was sent by Atlassian JIRA

View raw message