hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8549) Allow other implementation's of java.util.Map in MapWritable
Date Sun, 15 Jul 2012 16:10:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414687#comment-13414687
] 

Harsh J commented on HADOOP-8549:
---------------------------------

Hi and thanks for the reply!

bq. I tried it to make as ArrayWritable but it breaks the existing MapWritable code since
implementation class information has to be written into stream. Even i tried setters/getters
but it will not work since in M/R code Writables are created through default constructor.
So protected member was only way i can think of solving the problem.

{code}
/** {@inheritDoc} */
  @Override
  public void write(DataOutput out) throws IOException {
    super.write(out);
    
    // Write out the number of entries in the map
    
    out.writeInt(instance.size());

    // Then write out each key/value pair
    
    for (Map.Entry<Writable, Writable> e: instance.entrySet()) {
      out.writeByte(getId(e.getKey().getClass()));
      e.getKey().write(out);
      out.writeByte(getId(e.getValue().getClass()));
      e.getValue().write(out);
    }
  }
{code}

In the above writer method, I do not see the instance's type being serialized. The key and
value types are indeed stored, but not of the instance's.

Why can we hence not have a {{MapWritable(Class <? extends Map<Writable, Writable>>
mapType)}} that then instantiates the class for the field member "instance", rather than a
dev hacking like that?

bq. Since client has to use different class to get the new behavior (ex LinkedHashMapWritable)
and Writables are not polymorphic, portability should not be an issue.

Lets still have a test case. Your current test case merely checks the ordering of a sequential
implementation without serializing it. Lets instead serialize it, deserialize it again with
your custom writable, check the ordering. Also deserialize it with a regular instance, and
check the content. Let me know if this makes sense. This will help catch regressions in future
and good tests are always good to have.

Thanks and let me know if am wrong somewhere or if you have any other questions! :)
                
> Allow other implementation's of java.util.Map in MapWritable
> ------------------------------------------------------------
>
>                 Key: HADOOP-8549
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8549
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 2.0.0-alpha
>            Reporter: madhukara phatak
>            Assignee: madhukara phatak
>            Priority: Minor
>         Attachments: HADOOP-8549.patch
>
>
> Current implementation of MapWritable uses HashMap as Map implementation. But in some
of the use cases we need other implementations of Map like LinkedHashMap,SortedMap.This jira
changes visibility of 'instance' in MapWritable from private to protected which allows us
to inject custom Map implementation through sub classing MapWritable  .  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message