hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1915) adding counters methods using String (as opposed to Enum)
Date Mon, 24 Mar 2008 12:49:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581528#action_12581528

Tom White commented on HADOOP-1915:

I've just profiled the patch to compare it to trunk and I don't see any speed penalty.

I ran a sort in the local job runner and got 118.4s with trunk vs. 118.6s with the patch,
averaged across 5 runs. Using a sampling profiler, I measured 5ms of time in the Counters
class with trunk, and <1ms with the patch.

bq. Of course, just adding the string-based method in the reporter with the ids would have
been even easier.

I wanted to keep the ordinal out of the public API, to make it easier to use, and potentially
less error-prone (mixing up ordinals).

bq. tests included -1. The patch doesn't appear to include any new or modified tests.
bq. Please justify why no tests are needed for this patch.

The changes are tested by existing counters tests.

> adding counters methods using String (as opposed to Enum)
> ---------------------------------------------------------
>                 Key: HADOOP-1915
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1915
>             Project: Hadoop Core
>          Issue Type: New Feature
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Tom White
>            Priority: Minor
>             Fix For: 0.17.0
>         Attachments: hadoop-1915.patch
> Currently to use the counters from within Map/Reduce code Enums have to be used, the
Enum class defines the group and the Enum itself the counter. Internally they are converted
to Strings (the class name and the enum toString) and you can retrieve them as strings from
the client API.
> Using dynamic counters (driven by configuration of the map/reduce) is not easy with the
counters Enum based API. For example, currently I have an Enum class with 50 enums and we
have to map the cardinality to the counter name on the client. This is cumbersome.
> This could be easily improve by adding a String based counter method increment(String
group, String counter, long count) to allow use of the counters without Enums.
> Internally this method already exists, so the changes are minimal.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message