hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Baff <Aaron.B...@telescope.tv>
Subject Writing an Hbase Result object out to SequenceFileOutputFormat
Date Tue, 25 Oct 2011 05:33:00 GMT
So, I'm trying to write out an Hbase Result object (same one I get from my TableMapper) to
a SequenceFileOutputFormat from my Reducer as the value, but I'm getting an error when it's
trying to get a serializer. It looks like the SerializationFactory can't find a Serialization
(only one listed in the Job's configuration in io.serializations is WritableSerialization)
that will accept the Result object. Which is funny, because from the source code of the WritableSerialization.accept(),
it uses Writable.class.isAssignableFrom(c). When I run that manually and pass it Result.class,
it returns true.

When I use TextOutputFormat, it outputs fine, which I'm guessing is because it just calls
.toString() on the Key & Value, so doesn't need to mess with the Serialization stuff.
So, any ideas or hints to try and get this working?

In case it matters, I'm on CDH3u1.

        at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
        at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:908)
        at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:393)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:354)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:427)
        at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:61)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:559)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)

View raw message