hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3970) Counters written to the job history cannot be recovered back
Date Thu, 04 Sep 2008 19:39:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628439#action_12628439

Hemanth Yamijala commented on HADOOP-3970:


- I think the {{escapeString(String str, char escapeChar, char charToEscape)}} API is still
useful by itself, and is easier to use in cases where the user has only one char to escape.
So, do we really need to deprecate this ?
- charHasPreEscape is not needed. It is always going to be the escape char. This is a bug
in the previous code too.


- The token delimiters can be declared final.


- equals and hashCode implementation: Since the counter's value can change over time, I am
thinking that these methods should not depend on the value of the counter. Needs to be thought
out a bit more.
- makeEscapedCompactString - use StringBuffer, instead of appending spaces.
- makeEscapedCompactString - do we need the spaces ? I think the expression is compact without
them. If we need them, then a space is missing when the counter closes.

- equals: Code is incorrectly returning true if classes don't match. 
- hashCode: Are there standard ways in which we can combine hashCodes of various implementations.
Also, the hashCode should not use the toString method to get the hashCode.
makeCompactEscapedString - names are different between similar API in Counter. Can we have
them uniform ? Also, you can use StringBuffer without any String concatanation at all. Same
comment on blank spaces.

- getBlock: We expect a start and end. If there's no start or no end, should we fail fast
? I guess today if the end does not come as expected, it returns the remaining String as the
- Same comments here as with the Group's implementation for overriding equals and hashcode.


- testCounter need not check the mainCounter and copyCounter's equality before getting them
from the strings. In fact, why do we need copyCounter at all ?

> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>                 Key: HADOOP-3970
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3970
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>         Attachments: HADOOP-3970-v1.patch, HADOOP-3970-v2.patch, HADOOP-3970-v3.patch
> Counters that are written to the JobHistory are stringified using {{Counters.makeCompactString()}}.
The format in which this api converts the counter into a string is _groupname.countername:value_.
The problem is that _groupname_ and _countername_ can contain a '.' and hence recovering the
counter becomes difficult. Since JobHistory can be used for various purposes, reconstructing
the counter object back might be useful. One such usecase is HADOOP-3245. There should be
some way to recover the counter object back from its string representation and also to keep
the string version readable.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message