hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Output.collect uses toString for custom key class. Is it possible to change this?
Date Thu, 18 Dec 2008 07:06:51 GMT

On Dec 16, 2008, at 9:30 AM, David Coe wrote:

> Since the SequenceFileOutputFormat doesn't like nulls, how would I use
> NullWritable?  Obviously output.collect(key, null) isn't working.   
> If I
> change it to output.collect(key, new IntWritable()) I get the result I
> want (plus an int that I don't), but output.collect(key, new
> NullWritable()) does not work.

Sorry, I answered you literally. You can write a SequenceFile with  
NullWritables as the values, but you really want optional nulls. I'd  
probably define a Wrapper class like GenericWritable. It would look  
something like:

class NullableWriable<T extends Writable> implements Writable {
   private T instance;
   private boolean isNull;
   public void setNull(boolean isNull) {
     this.isNull = isNull;
   }
   public void readFields(DataInput in) throws IOException {
     read isNull;
     if (!isNull) {
        instance.readFields(in);
   }
   public void write(DataOutput out) throws IOException {
     write isNull;
     if (!isNull) {
        instance.write(out);
     }
   }
}

-- Owen

Mime
View raw message