ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-6808) Ambari Agent DataCleaner should delete log files when a max size in MB is reached
Date Mon, 11 Aug 2014 03:15:11 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alejandro Fernandez updated AMBARI-6808:
----------------------------------------

    Description: 
The log files in /var/log/ambari-agent can sometimes quickly fill up VMs. The current policy
allows running the DataCleaner every x hours (typically 1), and delete files older than y
days (typically 30). But this is not sufficient when a VM can only allocate say 100 MB to
store log files.

There is one bug that structured-out-####.json and command-####.json files are not currently
deleted by the DataCleaner.

The other ask is to allow deleting the oldest files first when the total space used for log
files exceeds z MB (typically 100, but can be as high as 10 GB).

One solution is to modify DataCleaner.py so that it keeps track of data_cleanup_max_size_MB.
If after the first round of deletes, the total space used by log files is greater than data_cleanup_max_size_MB,
then start deleting the oldest files first until the total size is less than the cap.

  was:
The log files in /var/log/ambari-agent can sometimes quickly fill up VMs. The current policy
allows running the DataCleaner every x hours (typically 1), and delete files older than y
days (typically 30). But this is not sufficient when a VM can only allocate say 100 MB to
store log files.

There is one bug that structured-out-####.json and command-####.json files are not currently
deleted by the DataCleaner.

The other ask is to allow deleting the oldest files first when the total space used for log
files exceeds z MB (typically 100, but can be as high as 10 GB). 


> Ambari Agent DataCleaner should delete log files when a max size in MB is reached
> ---------------------------------------------------------------------------------
>
>                 Key: AMBARI-6808
>                 URL: https://issues.apache.org/jira/browse/AMBARI-6808
>             Project: Ambari
>          Issue Type: Improvement
>          Components: agent
>    Affects Versions: 1.6.0
>            Reporter: Alejandro Fernandez
>            Assignee: Alejandro Fernandez
>             Fix For: 1.7.0
>
>
> The log files in /var/log/ambari-agent can sometimes quickly fill up VMs. The current
policy allows running the DataCleaner every x hours (typically 1), and delete files older
than y days (typically 30). But this is not sufficient when a VM can only allocate say 100
MB to store log files.
> There is one bug that structured-out-####.json and command-####.json files are not currently
deleted by the DataCleaner.
> The other ask is to allow deleting the oldest files first when the total space used for
log files exceeds z MB (typically 100, but can be as high as 10 GB).
> One solution is to modify DataCleaner.py so that it keeps track of data_cleanup_max_size_MB.
If after the first round of deletes, the total space used by log files is greater than data_cleanup_max_size_MB,
then start deleting the oldest files first until the total size is less than the cap.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message