hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-7021) MapWritable NullPointerException
Date Fri, 08 May 2015 21:37:04 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-7021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ravi Prakash updated HADOOP-7021:
    Attachment: HADOOP-7021.02.patch

Thanks a lot [~jlee@mindset-media.com] and [~tpierce] for your work on this issue! I'm sorry
this has been neglected for so long.

This seems like a legitimate issue. Its unfortunate that we've labeled this class Public and
Stable. :( Maybe we will have to put this in trunk only as a backward incompatible change.

I wonder whether the fix is to reconcile newClasses (only on an add) as in this patch, or
to actually never let it diverge. Specially because we are also not marking getNewClasses()
as private. Here it seems to be diverging because readFields doesn't do "the right thing"
. Although it is never explicitly documented, readFields seems to be expected to clear out
the original data in the object. Is my understanding correct?

> MapWritable NullPointerException
> --------------------------------
>                 Key: HADOOP-7021
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7021
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.20.1, 0.21.0
>         Environment: Hadoop 0.20.1, Centos
>            Reporter: John Lee
>            Assignee: John Lee
>            Priority: Minor
>              Labels: BB2015-05-RFC
>         Attachments: HADOOP-7021.02.patch, HADOOP-7021.patch, HADOOP-7021.patch, HADOOP-7021.patch,
> We have encountered a NullPointerException when we use MapWritable with custom Writable
> The root cause is the counter newClasses in AbstractMapWritable is allowed to get out
of sync with the id to class mapping tables when addToMap(Class) is called.  We have a patch
to AbstractMapWritable.addToMap(Class) that handles the case when newClasses gets out of sync
with the id to class mapping tables and adds a serialization optimization to minimize the
number of class names to write out per MapWritable object.

This message was sent by Atlassian JIRA

View raw message