hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2244) MapWritable.readFields needs to clear internal hash else instance accumulates entries forever
Date Wed, 21 Nov 2007 19:22:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544574

Hadoop QA commented on HADOOP-2244:

-1 overall.  Here are the results of testing the latest attachment 
against trunk revision r597144.

    @author +1.  The patch does not contain any @author tags.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new compiler warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests -1.  The patch failed core unit tests.

    contrib tests -1.  The patch failed contrib unit tests.

Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1136/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1136/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1136/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1136/console

This message is automatically generated.

> MapWritable.readFields needs to clear internal hash else instance accumulates entries
> ---------------------------------------------------------------------------------------------
>                 Key: HADOOP-2244
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2244
>             Project: Hadoop
>          Issue Type: Bug
>          Components: io
>            Reporter: stack
>             Fix For: 0.16.0
>         Attachments: hadoop-2244.patch
> A common framework pattern is to get an instance of a Writable, usually by reflection,
and then just keep calling readFields to make new 'instances' of the particular Writable.
> For example, the spill-to-disk that is run at the end of a map task gets instances of
map output keys and values and then loops over the (sorted) map output calling readFields
to make instances to write out to the filesystem (See around line #470 in the spill method).
> If the particular Writable is an instance of MapWritable, currently we get funny results.
 It has an internal hash map that is created on instantiation.  Each time the readFields method
is called, the newly deserialized entries are added to the internal map.  The map needs to
be reset when readFields is called so it doesn't just keep growing ad infinitum.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message