hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4148) MapReduce should not have a compile-time dependency on HDFS
Date Fri, 04 May 2012 20:50:50 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268698#comment-13268698
] 

Daryn Sharp commented on MAPREDUCE-4148:
----------------------------------------

Just a question: is there an advantage to using google's maps?  Given this jira's title, it
seems ironic to add a dep. :)

I've been discouraged from using static class init blocks due to the wonky exceptions it may
cause. I'd consider having something like a static "getClassForIdentifier" that populates
a map on first call from {{decodeIdentifier}}.

I missed it before, but you can pass a null conf to {{ReflectionUtils.newInstance}}.  It shorts
out trying to set the conf when null, and it also avoids instantiating unnecessary objects.
 Configs also hang around in a weak hash map for awhile...

Mainly because redundancy drives me nuts, but also because some errant exception could cause
{{addBinaryBuffer}} to be called twice, would you consider this? 
{code}
  private void identifierToString(StringBuilder buffer) {
    T id = null;
    try {
      id = decodeIdentifier();
    } finally {
      if (id != null) {
        buffer.append("(").append(id).append(")");
      } else {
        addBinaryBuffer(buffer, identifier);
      }
    }
  }
{code}
                
> MapReduce should not have a compile-time dependency on HDFS
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-4148
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4148
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Tom White
>            Assignee: Tom White
>         Attachments: MAPREDUCE-4148.patch, MAPREDUCE-4148.patch, MAPREDUCE-4148.patch,
MAPREDUCE-4148.patch, MAPREDUCE-4148.patch
>
>
> MapReduce depends on HDFS's DelegationTokenIdentifier (for printing token debug information).
We should remove this dependency and MapReduce's compile-time dependency on HDFS.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message