hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Macdonald (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3753) metrics: FileContext support overwrite mode
Date Mon, 14 Jul 2008 20:14:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613437#action_12613437

Craig Macdonald commented on HADOOP-3753:

I'm open to enlarging the requirements of this issue.

However, my 2c is that I want to have a scoreboard file that I can easily point a script at
to read (in this case collect and feed into RRD tool). 

IMHO, Log rotations etc can be handled externally - e.g. see logrotate(8) on linux. Overwriting
should not loose data as the file is a scoreboard - it has current values of the counters.
The metrics should continue to emit updated values, the client application reads the scoreboard
file when it feels the need (and possibly store the values in a time-indexed format externally).

This is how the counters etc in /proc on linux work. For example, /proc/net/snmp is read every
5 minutes. It's one (virtual) scoreboard file, which is kept updated by the kernel (but it
could have equally been implemented as being dumped out once a second/minute etc).

> metrics: FileContext support overwrite mode
> -------------------------------------------
>                 Key: HADOOP-3753
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3753
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Craig Macdonald
>            Priority: Minor
> FileContext currently continually appends to the metrics log file(s), generating an ever
lengthening file.
> In some scenarios, it would be useful to simply write the current statistics to the file
once every period, then overwrite the file for the next period.
> For instance, this could be useful if an external application parsed the metrics output
- e.g. Cacti to create realtime graphs.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message