airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-108) Add data retention policy to Airflow
Date Thu, 31 Aug 2017 01:18:00 GMT


ASF subversion and git services commented on AIRFLOW-108:

Commit de593216d9dbe8e99f572d9cb509e3b398ecfef6 in incubator-airflow's branch refs/heads/master
from Sid Anand
[;h=de59321 ]

[AIRFLOW-108] Add to companies list

Dear Airflow maintainers,

Please accept this PR. I understand that it will
not be reviewed until I have checked off all the
steps below!

### JIRA
- [/] My PR addresses the following [Airflow JIRA]
issues and references them in the PR title. For
example, "[AIRFLOW-XXX] My Airflow PR"

### Description
- [/] Here are some details about my PR, including
screenshots of any UI changes:
Adding an entry to the companies list in

### Tests
- [/] My PR adds the following unit tests __OR__
does not need testing for this extremely good
reason: Documentation change only.

### Commits
- [/] My commits all reference JIRA issues in
their subject lines, and I have squashed multiple
commits if they address the same issue. In
addition, my commits follow the guidelines from
"[How to write a good git commit
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Closes #2554 from r39132/master

> Add data retention policy to Airflow
> ------------------------------------
>                 Key: AIRFLOW-108
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: db, scheduler
>            Reporter: Chris Riccomini
> Airflow's DB currently holds the entire history of all executions for all time. This
is problematic as the DB grows. The UI starts to get slower, and the DB's disk usage grows.
There is no bound to how large the DB will grow.
> It would be useful to add a feature in Airflow to do two things:
> # Delete old data from the DB
> # Mark some lower watermark, past which DAG executions are ignored
> For example, (2) would allow you to tell the scheduler "ignore all data prior to a year
ago". And (1) would allow Airflow to delete all data prior to January 1, 2015.

This message was sent by Atlassian JIRA

View raw message