airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Clark (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRFLOW-6435) Mount secret files from secrets config
Date Fri, 03 Jan 2020 14:40:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007248#comment-17007248
] 

Brandon Clark edited comment on AIRFLOW-6435 at 1/3/20 2:39 PM:
----------------------------------------------------------------

I have created an implementation which requires the following entry in airflow.cfg which will
reference a Kubernetes secret configuration for keys matching each filename and place them
in the corresponding folder:
{code:java}
[kubernetes]
secret_files_secret = airflow
secret_files_dir = /root/.ssh
secret_files_filenames = airflow.pem,client.key{code}
 

The generated output results in the following additions to a pod configuration:
{code:java}
apiVersion: v1
kind: Pod
metadata:
  name: example-dag
spec:
  containers:
    volumeMounts:
    - mountPath: /root/.ssh/airflow.pem
      name: airflow-secret-files
      subPath: airflow.pem
    - mountPath: /root/.ssh/client.key
      name: airflow-secret-files
      subPath: client.key
  volumes:
  - name: airflow-secret-files
    secret:
      defaultMode: 256
      items:
      - key: airflow.pem
        path: airflow.pem
      - key: client.key
        path: client.key
      secretName: airflow{code}
 

If this seems acceptable I can make a commit to 1.10.x and 2.0.x code bases.  Any suggestions
before I attempt to make a commit request?


was (Author: webmind):
I have created an implementation which requires the following entry in airflow.cfg which will
reference a Kubernetes secret configuration for keys matching each filename and place them
in the corresponding folder:
{code:java}
[kubernetes]
secret_file_secret = airflow
secret_file_dir = /root/.ssh
secret_file_filenames = airflow.pem,client.key{code}
 

The generated output results in the following additions to a pod configuration:
{code:java}
apiVersion: v1
kind: Pod
metadata:
  name: example-dag
spec:
  containers:
    volumeMounts:
    - mountPath: /root/.ssh/airflow.pem
      name: airflow-secret-files
      subPath: airflow.pem
    - mountPath: /root/.ssh/client.key
      name: airflow-secret-files
      subPath: client.key
  volumes:
  - name: airflow-secret-files
    secret:
      defaultMode: 256
      items:
      - key: airflow.pem
        path: airflow.pem
      - key: client.key
        path: client.key
      secretName: airflow{code}
 

If this seems acceptable I can make a commit to 1.10.x and 2.0.x code bases.  Any suggestions
before I attempt to make a commit request?

> Mount secret files from secrets config
> --------------------------------------
>
>                 Key: AIRFLOW-6435
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6435
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: executor-kubernetes
>    Affects Versions: 2.0.0, 1.10.8
>            Reporter: Brandon Clark
>            Assignee: Brandon Clark
>            Priority: Major
>              Labels: Kubernetes, executor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> There should be a dynamic way to add protected files to pod.  Just as git-sync requires
an ssh key to mounted so do other software suites and processes that can be ran from Airflow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message