hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Hoffman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-6244) Improvements to FileContext metrics output formatting
Date Mon, 08 Aug 2011 15:42:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081007#comment-13081007

Steve Hoffman commented on HADOOP-6244:

I'd like to argue that this should be routed thru log4j so we can use something like RollingFileAppender
to deal with the long term consequences of the file being written to forever.
Then the configurable part is just what format you want the record.  Again, can take a hint
from log4j by providing %LETTER parameters for specifying that.

The root of the problem I see here is that any simple 'grep' log message by definition has
to have 1 line per record with a newline.  Thus, you can't have a structured message in there
w/o doing some kind of encoding which leads you to avro/protobufs/etc. which make it not usable
in grep (usually).  Seems you can't have it both ways.

This doesn't apply so much to metrics as you can just use key/value pairs on 1 line, but would
still like to see this more general so if I WANTED protobufs/avro/etc. I would just implement
the record formatter part and output a 'string'

> Improvements to FileContext metrics output formatting
> -----------------------------------------------------
>                 Key: HADOOP-6244
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6244
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Steve Loughran
>         Attachments: hadoop-6244.txt
> The output of FileContext has two big issues: 1) it doesn't include a timestamp, 2) it
doesn't differentiate between tags and metrics in formatting. This patch is to improve the
output format to be more useful.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message