airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Standish <dpstand...@gmail.com>
Subject Re: Backing up PostgreSQL DB on K8s
Date Wed, 09 Jun 2021 05:46:00 GMT
Perhaps it goes without saying but you might consider using cloud sql
option such as aws rds, which provides persistence even if you destroy and
rebuild your k8s cluster, and of course automated backups.

On Tue, Jun 8, 2021, 10:41 PM Sumit Maheshwari <msumit@apache.org> wrote:

> If you are backing up data to safeguard against pod failures, then I
> believe that you can use a PV as data storage for PSql & it would survive
> any pod restarts.
>
> On Wed, Jun 9, 2021 at 5:01 AM Lewis John McGibbney <lewismc@apache.org>
> wrote:
>
>> Hi users@,
>> Does anyone have a recommended/best practice/preferred way for backing up
>> PostgreSQL when Airflow is deployed into K8s?
>> We were thinking of writing a maintenance DAG which would do this...
>> maybe even contributing it to
>> https://github.com/teamclairvoyant/airflow-maintenance-dags.
>> I'm thinking it would just authenticate into the K8s cluster, find the
>> postgresql pod and then perform a DB archival in s3 or something.
>> I also looked more into
>> https://www.postgresql.org/docs/9.1/continuous-archiving.html which
>> looks appealing. I'm interest to see what others are doing.
>> Any suggestions appreciated.
>> Thank you
>> lewismc
>>
>

Mime
View raw message