Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 27 Jul 2011 15:07:09 +0000 (UTC)
From: "Jonathan Ellis (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <1895372946.12180.1311779229831.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <104166224.11673.1311769390275.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (CASSANDRA-2955) Hadoop cannot read CounterColumn
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-2955:
--------------------------------------

             Priority: Minor  (was: Major)
    Affects Version/s:     (was: 0.8.1)
                       0.8.0
        Fix Version/s: 0.8.3
               Labels: lhf  (was: )
           Issue Type: New Feature  (was: Bug)

> Hadoop cannot read CounterColumn
> --------------------------------
>
>                 Key: CASSANDRA-2955
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Hadoop
>    Affects Versions: 0.8.0
>            Reporter: Jacek Gerbszt
>            Priority: Minor
>              Labels: lhf
>             Fix For: 0.8.3
>
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> 	at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> 	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> 	at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
>         private IColumn unthriftify(ColumnOrSuperColumn cosc)
>         {
>             if (cosc.column == null)
>                 return unthriftifySuper(cosc.super_column);
>             return unthriftifySimple(cosc.column);
>         }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
>     public org.apache.cassandra.thrift.Column column;
>     public org.apache.cassandra.thrift.SuperColumn super_column;
>     public org.apache.cassandra.thrift.CounterColumn counter_column;
>     public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordReader class.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira