hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6244) Improvements to FileContext metrics output formatting
Date Tue, 03 Nov 2009 03:51:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772867#action_12772867
] 

Todd Lipcon commented on HADOOP-6244:
-------------------------------------

bq. While timestamps and distinguishing tags from metrics are both useful properties, changing
the format of FileContext disrupts all the downstream consumers.

Agreed - this is why I left the old format in as a configuration option. Is anyone actually
*using* the old format, though? When I wanted to look at metrics logs from FileContext in
the past, I ended up actually writing a perl script to interpolate timestamps based on the
file's mtime - pretty awful. I think now (ie before 1.0) is the time when we should feel free
to change formats/APIs that are clearly bad as long as we provide a deprecated compatibility
path, yes?


As for the code notes, I think all of your points are valid - I'll upload a new patch soon.

bq. better served by adding a FileContext using a standard format, like JSON, or base it on
Avro.

JSON would be reasonable, but I think it's important we continue to have a simpler text metrics
logging option. When fishing around on nodes it's good to be able to use perl, grep, and awk
without having to install a JSON parser. Avro makes sense for those who want to do long term
analysis, but I think that should be a separate patch.

> Improvements to FileContext metrics output formatting
> -----------------------------------------------------
>
>                 Key: HADOOP-6244
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6244
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-6244.txt
>
>
> The output of FileContext has two big issues: 1) it doesn't include a timestamp, 2) it
doesn't differentiate between tags and metrics in formatting. This patch is to improve the
output format to be more useful.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message