hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Soztutar <enis.soz.nu...@gmail.com>
Subject Re: Why ComparableWritable does not take a template?
Date Mon, 12 May 2008 15:33:58 GMT
Hi,

WritableComparable uses generics in trunk, but if you use 0.16.x you 
cannot use that version. WritableComparable is not generified yet due to 
legacy reasons, but the work is in progress. The problem with your code 
is raising from WritableComparator.newKey(). It seems your object cannot 
be created by keyClass.newInstance() (no default constructor I guess). 
You can implement you own WritableComparator implementation.

steph wrote:
>
> The example in the java doc shows that the compareTo() method uses the 
> type of the class instead of the
> Object type. However the ComparableWritable class does not take any 
> template and therefore it cannot
> set the tempate for the class Comparable. Is that a mistake?
>
> *     public class MyWritableComparable implements WritableComparable {
>  *       // Some data
>  *       private int counter;
>  *       private long timestamp;
>  *
>  *       public void write(DataOutput out) throws IOException {
>  *         out.writeInt(counter);
>  *         out.writeLong(timestamp);
>  *       }
>  *
>  *       public void readFields(DataInput in) throws IOException {
>  *         counter = in.readInt();
>  *         timestamp = in.readLong();
>  *       }
>  *
>  *       public int compareTo(MyWritableComparable w) {
>  *         int thisValue = this.value;
>  *         int thatValue = ((IntWritable)o).value;
>  *         return (thisValue &lt; thatValue ? -1 : 
> (thisValue==thatValue ? 0 : 1));
>  *       }
>  *     }
>
> If i try to do what it example shows it does not compile:
>
>   [javac] 
> /Users/steph/Work/Rinera/TRUNK/vma/hadoop/parser-hadoop/apps/src/com/rinera/hadoop/weblogs/SummarySQLKey.java:13:

> com.rinera.hadoop.weblogs.SummarySQLKey is not abstract and does not 
> override abstract method compareTo(java.lang.Object) in 
> java.lang.Comparable
>     [javac] public class SummarySQLKey
>
>
> If i don't use the type but instead use the Object type for compareTo()
> i get a RuntimeException:
>
> java.lang.RuntimeException: java.lang.InstantiationException: 
> com.rinera.hadoop.weblogs.SummarySQLKey
>     at 
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:75) 
>
>     at 
> org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:63) 
>
>     at 
> org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:42)
>     at 
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:645)
>     at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:313)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:174)
>     at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)
> Caused by: java.lang.InstantiationException: 
> com.rinera.hadoop.weblogs.SummarySQLKey
>     at java.lang.Class.newInstance0(Class.java:335)
>     at java.lang.Class.newInstance(Class.java:303)
>     at 
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:73) 
>
>     ... 6 more
>
>
>
>

Mime
View raw message