Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 05122200CCA for ; Wed, 19 Jul 2017 19:40:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 039901697DE; Wed, 19 Jul 2017 17:40:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4EA261697DA for ; Wed, 19 Jul 2017 19:40:03 +0200 (CEST) Received: (qmail 10230 invoked by uid 500); 19 Jul 2017 17:40:02 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 10221 invoked by uid 99); 19 Jul 2017 17:40:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Jul 2017 17:40:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 15ED6C041C for ; Wed, 19 Jul 2017 17:40:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id VCeS-is5IHSG for ; Wed, 19 Jul 2017 17:40:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 17C525FD9C for ; Wed, 19 Jul 2017 17:40:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 5CE4FE06CF for ; Wed, 19 Jul 2017 17:40:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 18FED21EA5 for ; Wed, 19 Jul 2017 17:40:00 +0000 (UTC) Date: Wed, 19 Jul 2017 17:40:00 +0000 (UTC) From: "Dennis Docter (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (AIRFLOW-1310) Kubernetes execute operator MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 19 Jul 2017 17:40:04 -0000 [ https://issues.apache.org/jira/browse/AIRFLOW-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093479#comment-16093479 ] Dennis Docter commented on AIRFLOW-1310: ---------------------------------------- It's similar to ours. We opted not to go with jobs, as kubernetes will then take care of scheduling, retries etc, while that should be the responsibility of Airflow in this case. Also you are not capturing any logs, preferably streaming, which would be a basic requirement for the operator (especially since the pods get deleted afterwards) > Kubernetes execute operator > --------------------------- > > Key: AIRFLOW-1310 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1310 > Project: Apache Airflow > Issue Type: New Feature > Components: operators > Reporter: Yu Ishikawa > Assignee: Dennis Docter > > We should support operators for Kubernetes to execute containers remotely. > - Create a Kubernetes cluster > - Execute PODs > - https://github.com/kubernetes-incubator/client-python -- This message was sent by Atlassian JIRA (v6.4.14#64029)