hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3970) Counters written to the job history cannot be recovered back
Date Thu, 04 Sep 2008 19:39:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628439#action_12628439
] 

Hemanth Yamijala commented on HADOOP-3970:
------------------------------------------

StringUtils

- I think the {{escapeString(String str, char escapeChar, char charToEscape)}} API is still
useful by itself, and is easier to use in cases where the user has only one char to escape.
So, do we really need to deprecate this ?
- charHasPreEscape is not needed. It is always going to be the escape char. This is a bug
in the previous code too.

Counters

- The token delimiters can be declared final.

Counter

- equals and hashCode implementation: Since the counter's value can change over time, I am
thinking that these methods should not depend on the value of the counter. Needs to be thought
out a bit more.
- makeEscapedCompactString - use StringBuffer, instead of appending spaces.
- makeEscapedCompactString - do we need the spaces ? I think the expression is compact without
them. If we need them, then a space is missing when the counter closes.

Group
- equals: Code is incorrectly returning true if classes don't match. 
- hashCode: Are there standard ways in which we can combine hashCodes of various implementations.
Also, the hashCode should not use the toString method to get the hashCode.
makeCompactEscapedString - names are different between similar API in Counter. Can we have
them uniform ? Also, you can use StringBuffer without any String concatanation at all. Same
comment on blank spaces.

Counters:
- getBlock: We expect a start and end. If there's no start or no end, should we fail fast
? I guess today if the end does not come as expected, it returns the remaining String as the
value.
- Same comments here as with the Group's implementation for overriding equals and hashcode.

TestCounters:

- testCounter need not check the mainCounter and copyCounter's equality before getting them
from the strings. In fact, why do we need copyCounter at all ?


> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>
>                 Key: HADOOP-3970
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3970
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>         Attachments: HADOOP-3970-v1.patch, HADOOP-3970-v2.patch, HADOOP-3970-v3.patch
>
>
> Counters that are written to the JobHistory are stringified using {{Counters.makeCompactString()}}.
The format in which this api converts the counter into a string is _groupname.countername:value_.
The problem is that _groupname_ and _countername_ can contain a '.' and hence recovering the
counter becomes difficult. Since JobHistory can be used for various purposes, reconstructing
the counter object back might be useful. One such usecase is HADOOP-3245. There should be
some way to recover the counter object back from its string representation and also to keep
the string version readable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message