hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5228) Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and mapreduce
Date Fri, 24 May 2013 18:30:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13666524#comment-13666524
] 

Zhijie Shen commented on MAPREDUCE-5228:
----------------------------------------

1. FileInputFormat.Counter and FileOutputFormat.Counter in mapreduce should be added as well.

2. depricatedCounterMap, initDepricatedMap() and getNewGroupKey() is better to be defined
in AbstractCounters

3. Whenever FileInputFormatCounter and FileOutputFormatCounter are referred, the mapping policy
needs to be applied. Now it's only applied in one place.

4. Is it simpler to do the following in AbstractCounters:
{code}
...
  public synchronized C findCounter(Enum<?> key) {
+   key = oldToNew(key);
    C counter = cache.get(key);
...
+ private Enum<?> oldToNew(Enum<?> key) {
+   if (key == FileInputFormat.Counter) {
+     return FileInputFormatCounter;
+   } else if (key == FileOutputFormat.Counter) {
+     return FileOutputFormatCounter
+   }
+   // more mapping rules
+   return key;
+ }
...
{code}

AbstractCounters#findCounter seems to be the ultimate method to be called. Moreover, since
there won't be a lot of mapping rules, the sequential lookup through "if else" should be okay.
                
> Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and
mapreduce
> ----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5228
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5228
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Mayank Bansal
>         Attachments: MAPREDUCE-5228-trunk-1.patch
>
>
> The enum was used by findCounter(Enum key) to find a specific counter object. Now it
seems to be replaced by FileInputFormatCounter and FileOutputFormatCounter. Now the enum seems
to be only used internally, but not sure whether it will be used externally when users extend
FileXXXXFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message