hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2997) Add test for non-writable serializer
Date Thu, 13 Mar 2008 11:35:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578250#action_12578250
] 

Tom White commented on HADOOP-2997:
-----------------------------------

Tracked down the problem - the change in HADOOP-2399 creates instances of keys and values
to be reused, however this is not possible for Java serialization since the types don't necessarily
have no-arg constructors:

java.lang.RuntimeException: java.lang.NoSuchMethodException: java.lang.Long.<init>()
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:80)
        at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.<init>(ReduceTask.java:213)
        at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.<init>(ReduceTask.java:306)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:388)
        at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)
Caused by: java.lang.NoSuchMethodException: java.lang.Long.<init>()
        at java.lang.Class.getConstructor0(Class.java:2678)
        at java.lang.Class.getDeclaredConstructor(Class.java:1953)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:74)
        ... 4 more

The fix is to get the deserializer to create the key and value types, if needed. I'm testing
a patch.

> Add test for non-writable serializer
> ------------------------------------
>
>                 Key: HADOOP-2997
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2997
>             Project: Hadoop Core
>          Issue Type: Test
>          Components: mapred
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 0.17.0
>
>         Attachments: hadoop-2997-v2.patch, hadoop-2997.patch
>
>
> It would be useful to have a unit test that tests MapReduce works using a non-writable
serializer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message