ace-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ronald Spierenburg (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACE-378) Truncate server logs.
Date Wed, 30 Oct 2013 20:41:29 GMT

     [ https://issues.apache.org/jira/browse/ACE-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ronald Spierenburg updated ACE-378:
-----------------------------------

    Attachment: ACE-378.patch

During several discussions with Marcel regarding this subject we discussed two different solutions
to this problem. 

Syncing : 
1) Tell the other targets/servers in the sync process how many log-lines we would want to
have. This would limit data traffic and limit the disk-size used on the server for logs.
2) Don't tell the other party of the limit and remove what we don't need.

Option 2 was deemed the most appropriate because option 1 would require a new protocol version
(and all the problems that come with that). 

Storage :
1) Limit the total disk-size used for logs
2) Limit the size of the logs of a per target
3) Limit the number of events of a per log for a per target

At first a combination of option 1+2 was chosen because this would make sure that the amount
of allocated disk-space would not be exceeded and that there wouldn't be gateways 'hogging'
the store.
Processing all the logs for a single target synchronously on every put operation would however
have a significant negative impact on performance and therefore this option is discarded.


In the end solution 3 was picked as a solution that could be a) fast and b) easily implemented.
This off course means that there could still be lots of disk-space used, potentially without
limits. This risk is deemed minimal. When scaling ACE up the amount of data in an auditlog
can be estimated. Other channels can not be predicted by ace but the amount of data can be
estimated by the users of that channel. 
The locking strategy for the logstore (synchronized public operations) is changed to a more
refined locking mechanism based on the store for the target to speed up parallel log processing.


Solution
Sync option 2 with storage option 3 are implemented in the enclosed patch. To facilitate changed
configurations (and the extra cleanup they may entail) a gogo-shell command is included that
will manually perform a cleanup according to the newly configured maximum size.


> Truncate server logs.
> ---------------------
>
>                 Key: ACE-378
>                 URL: https://issues.apache.org/jira/browse/ACE-378
>             Project: ACE
>          Issue Type: Improvement
>          Components: Log
>            Reporter: Marcel Offermans
>            Assignee: Angelo van der Sijpt
>         Attachments: ACE-378.patch
>
>
> We need a configurable way to truncate the logs that are stored on the server. At least
we should be able to limit them to some maximum size.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message