hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3665) WritableComparator newKey() fails for NullWritable
Date Mon, 30 Jun 2008 22:00:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609381#action_12609381
] 

Chris Douglas commented on HADOOP-3665:
---------------------------------------

* NullWritable should define a RawComparator, so it doesn't need to use the default WritableComparator
rather than special-casing NullWritable
* Your apt use of ReflectionUtils calls out a separate issue with WritableComparator, i.e.
that the keys and values it uses are not Configured, where in most contexts they are. This
should probably be a separate JIRA...
* SequenceFile shouldn't make a special case of NullWritable, either. Does the rest of the
logic still work if the expression is changed from keylength == 0 to keylength < 0?
* ReflectionUtils shouldn't special-case NullWritable, either. Are you certain this is necessary?
Doesn't setAccessible take care of this?

> WritableComparator newKey() fails for NullWritable
> --------------------------------------------------
>
>                 Key: HADOOP-3665
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3665
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.17.0
>         Environment: n/a
>            Reporter: Lukas Vlcek
>            Priority: Minor
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3665.path
>
>
> It is not possible to use NullWritable as a key in order to suppress key value in output.
> Syndrome exception:
> Caused by: java.lang.IllegalAccessException: Class org.apache.hadoop.io.WritableComparator
can not access a member of class org.apache.hadoop.io.NullWritable with modifiers "private"
> The problem is that NullWritable is a singleton and does not provide public non-parametric
constructor. The following code in WritableComparator causes the exception: return (WritableComparable)keyClass.newInstance();
> Proposed simple solution is to use ReflectionUtils instead (it requires modification
as well).
> This issue is probably related to HADOOP-2922

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message