hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7328) Give more information about a missing Serializer class
Date Thu, 16 Jun 2011 18:21:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050626#comment-13050626

Harsh J commented on HADOOP-7328:

I actually do have two comments of concern w.r.t. front-end & throwing:

If we are to make code changes _throw-y_, then it would require that all places that use the
said API inside Hadoop add throws IOException to their methods. Upon a quick check, only one
single SequenceFile private utility method required this change, and none in MapReduce. But
it could've been that the change required subsequent public API changes.

Since we've never had front end checks for MR serializers before; if people relied on passing
strings over to job.xml and have it resolve only on worker machines - the related issue may
break this functionality. Guideline-wise, its a wrong way to do it, but I think it still breaks
something deep below I think.

But I think its worth enforcing this check (null handling later or throw above, either way)
since it makes stuff like job setup errors quicker to debug and iterate upon.

> Give more information about a missing Serializer class
> ------------------------------------------------------
>                 Key: HADOOP-7328
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7328
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.20.2
>            Reporter: Harsh J
>            Assignee: Harsh J
>              Labels: io, serialization
>             Fix For: 0.23.0
>         Attachments: HADOOP-7328.r1.diff, HADOOP-7328.r2.diff, HADOOP-7328.r3.diff
> When you have a key/value class that's non Writable and you forget to attach io.serializers
for the same, an NPE is thrown by the tasks with no information on why or what's missing and
what led to it. I think a better exception can be thrown by SerializationFactory instead of
an NPE when a class is not found accepted by any of the loaded ones.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message