Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Wed, 20 Feb 2013 16:39:13 +0000 (UTC)
From: "Karthik Kambatla (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12631712.1360582625177.311107.1361378353191@arcas>
In-Reply-To: <JIRA.12631712.1360582625177@arcas>
References: <JIRA.12631712.1360582625177@arcas>
Subject: [jira] [Commented] (HADOOP-9295) AbstractMapWritable throws
 exception when calling readFields() multiple times when the maps contain
 different class types
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582307#comment-13582307 ] 

Karthik Kambatla commented on HADOOP-9295:
------------------------------------------

http://wiki.apache.org/hadoop/HowToContribute is a good place to start. The instructions there are for svn - you can definitely use git - http://wiki.apache.org/hadoop/GitAndHadoop

Note that Hadoop uses spaces for tabs - and both tab and indentation are 2 spaces.
                
> AbstractMapWritable throws exception when calling readFields() multiple times when the maps contain different class types
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-9295
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9295
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 1.0.3
>            Reporter: David Parks
>            Assignee: Karthik Kambatla
>            Priority: Critical
>         Attachments: MapWritableBugTest.java
>
>
> Verified the trunk looks the same as 1.0.3 for this issue.
> When mappers output MapWritables with different class types, then they are read in on the Reducer via an iterator (multiple calls to readFields without instantiating a new object) you'll get this:
> java.lang.IllegalArgumentException: Id 1 exists but maps to org.me.ClassTypeOne and not org.me.ClassTypeTwo
>         at org.apache.hadoop.io.AbstractMapWritable.addToMap(AbstractMapWritable.java:73)
>         at org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:201)
> It happens because AbstractMapWritable accumulates class type entries in its ClassType to ID (and vice versa) hashmaps.
> Those accumulating classtype-to-id hashmaps need to be cleared to support multiple calls to readFields().
> I've attached a JUnit test that both demonstrates the problem and contains an embedded, fixed version of MapWritable and ArrayMapWritable (note the //TODO comments in the code where it was fixed in 2 places).
> If there's a better way to submit this recommended bug fix, someone please feel free to let me know.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira