hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lukas Vlcek (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3665) WritableComparator newKey() fails for NullWritable
Date Sun, 29 Jun 2008 22:13:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lukas Vlcek updated HADOOP-3665:
--------------------------------

    Attachment: HADOOP-3665.path

Here is simple patch. Please review it!

Still there are few questions/notes opened I think:
1) There is no clear way how to learn whether particular WritableComparator can be instantiated
or not. There should be some new interface introduced for singletons.
2) It is not clear if key value can be of zero length. For NullWritable it make sense but
it has nothing to do with the fact that it is a singleton I think.
3) I found other two classes implementing WritableComparable having private or protected constructor.
These are ID and TaskID. Since they are both WritableComparable they may be used as a key
(I can not tell if there is any point in such use case but technically it should be possible).
This patch DOES NOT fixes such use case in any way.

Giving these notes together I can see the following conclusion:
Going forward it should be possible to learn if WritableComparable:
a) can be used as a key
b) can have zero length
c) can be instantiated using constructor and if not then what is the way to get the instance.

> WritableComparator newKey() fails for NullWritable
> --------------------------------------------------
>
>                 Key: HADOOP-3665
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3665
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.17.0
>         Environment: n/a
>            Reporter: Lukas Vlcek
>            Priority: Minor
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3665.path
>
>
> It is not possible to use NullWritable as a key in order to suppress key value in output.
> Syndrome exception:
> Caused by: java.lang.IllegalAccessException: Class org.apache.hadoop.io.WritableComparator
can not access a member of class org.apache.hadoop.io.NullWritable with modifiers "private"
> The problem is that NullWritable is a singleton and does not provide public non-parametric
constructor. The following code in WritableComparator causes the exception: return (WritableComparable)keyClass.newInstance();
> Proposed simple solution is to use ReflectionUtils instead (it requires modification
as well).
> This issue is probably related to HADOOP-2922

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message