Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: 209.17.190.200 is neither permitted
 nor denied by domain of enis.soz.nutch@gmail.com)
Message-ID: <48286366.7010200@gmail.com>
Date: Mon, 12 May 2008 18:33:58 +0300
From: Enis Soztutar <enis.soz.nutch@gmail.com>
User-Agent: Thunderbird 2.0.0.12 (X11/20080227)
MIME-Version: 1.0
To: core-user@hadoop.apache.org
Subject: Re: Why ComparableWritable does not take a template?
References: <AFABC011-C9F6-430A-9729-96F065971DA4@conviva.com>
In-Reply-To: <AFABC011-C9F6-430A-9729-96F065971DA4@conviva.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

WritableComparable uses generics in trunk, but if you use 0.16.x you 
cannot use that version. WritableComparable is not generified yet due to 
legacy reasons, but the work is in progress. The problem with your code 
is raising from WritableComparator.newKey(). It seems your object cannot 
be created by keyClass.newInstance() (no default constructor I guess). 
You can implement you own WritableComparator implementation.

steph wrote:
>
> The example in the java doc shows that the compareTo() method uses the 
> type of the class instead of the
> Object type. However the ComparableWritable class does not take any 
> template and therefore it cannot
> set the tempate for the class Comparable. Is that a mistake?
>
> *     public class MyWritableComparable implements WritableComparable {
>  *       // Some data
>  *       private int counter;
>  *       private long timestamp;
>  *
>  *       public void write(DataOutput out) throws IOException {
>  *         out.writeInt(counter);
>  *         out.writeLong(timestamp);
>  *       }
>  *
>  *       public void readFields(DataInput in) throws IOException {
>  *         counter = in.readInt();
>  *         timestamp = in.readLong();
>  *       }
>  *
>  *       public int compareTo(MyWritableComparable w) {
>  *         int thisValue = this.value;
>  *         int thatValue = ((IntWritable)o).value;
>  *         return (thisValue &lt; thatValue ? -1 : 
> (thisValue==thatValue ? 0 : 1));
>  *       }
>  *     }
>
> If i try to do what it example shows it does not compile:
>
>   [javac] 
> /Users/steph/Work/Rinera/TRUNK/vma/hadoop/parser-hadoop/apps/src/com/rinera/hadoop/weblogs/SummarySQLKey.java:13: 
> com.rinera.hadoop.weblogs.SummarySQLKey is not abstract and does not 
> override abstract method compareTo(java.lang.Object) in 
> java.lang.Comparable
>     [javac] public class SummarySQLKey
>
>
> If i don't use the type but instead use the Object type for compareTo()
> i get a RuntimeException:
>
> java.lang.RuntimeException: java.lang.InstantiationException: 
> com.rinera.hadoop.weblogs.SummarySQLKey
>     at 
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:75) 
>
>     at 
> org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:63) 
>
>     at 
> org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:42)
>     at 
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:645)
>     at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:313)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:174)
>     at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)
> Caused by: java.lang.InstantiationException: 
> com.rinera.hadoop.weblogs.SummarySQLKey
>     at java.lang.Class.newInstance0(Class.java:335)
>     at java.lang.Class.newInstance(Class.java:303)
>     at 
> org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:73) 
>
>     ... 6 more
>
>
>
>