hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudharsan Sampath (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7328) Give more information about a missing Serializer class
Date Wed, 15 Jun 2011 04:39:47 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049604#comment-13049604
] 

Sudharsan Sampath commented on HADOOP-7328:
-------------------------------------------

Just my thoughts...

To me throwing a specific exception would be better. The checkSerializerSpecs attempts to
see if we can initialise a new instance of the serializer/deserializer from the jvm where
the job is submitted. How does it guarantee that the libs/jars from which these classes are
loaded are available for the jvm that executes the job or vice versa? Even if this methods
succeeds isn't there a chance then the original problem might occur due to the libs missing
from the actual jvm that executes the job?



> Give more information about a missing Serializer class
> ------------------------------------------------------
>
>                 Key: HADOOP-7328
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7328
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.20.2
>            Reporter: Harsh J
>            Assignee: Harsh J
>              Labels: io, serialization
>             Fix For: 0.23.0
>
>         Attachments: HADOOP-7328.r1.diff, HADOOP-7328.r2.diff
>
>
> When you have a key/value class that's non Writable and you forget to attach io.serializers
for the same, an NPE is thrown by the tasks with no information on why or what's missing and
what led to it. I think a better exception can be thrown by SerializationFactory instead of
an NPE when a class is not found accepted by any of the loaded ones.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message