hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6244) Improvements to FileContext metrics output formatting
Date Tue, 03 Nov 2009 03:51:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772867#action_12772867

Todd Lipcon commented on HADOOP-6244:

bq. While timestamps and distinguishing tags from metrics are both useful properties, changing
the format of FileContext disrupts all the downstream consumers.

Agreed - this is why I left the old format in as a configuration option. Is anyone actually
*using* the old format, though? When I wanted to look at metrics logs from FileContext in
the past, I ended up actually writing a perl script to interpolate timestamps based on the
file's mtime - pretty awful. I think now (ie before 1.0) is the time when we should feel free
to change formats/APIs that are clearly bad as long as we provide a deprecated compatibility
path, yes?

As for the code notes, I think all of your points are valid - I'll upload a new patch soon.

bq. better served by adding a FileContext using a standard format, like JSON, or base it on

JSON would be reasonable, but I think it's important we continue to have a simpler text metrics
logging option. When fishing around on nodes it's good to be able to use perl, grep, and awk
without having to install a JSON parser. Avro makes sense for those who want to do long term
analysis, but I think that should be a separate patch.

> Improvements to FileContext metrics output formatting
> -----------------------------------------------------
>                 Key: HADOOP-6244
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6244
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>         Attachments: hadoop-6244.txt
> The output of FileContext has two big issues: 1) it doesn't include a timestamp, 2) it
doesn't differentiate between tags and metrics in formatting. This patch is to improve the
output format to be more useful.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message