cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <>
Subject Re: Hadoop Word Count Super Column Example?
Date Wed, 20 Oct 2010 08:27:15 GMT
My understanding of the Hadoop integration is not great but from what I can see. The code in
o.a.c.hadoop.ColumnFamilyRecordReader does not use a super_column in  the ColumnParent struct
when making the get_range_slices() call. It's just using the ColumnFamily. 

So I would guess it would include super columns if they were present. And that the IColumns
passed to your map function will be instances of o.a.c.db.SuperColumn. 

Have your tried it ?


On 20 Oct 2010, at 04:44, Jeremy Hanna wrote:

> It's relatively straightforward, the current mapper gets a map of column names to IColumns.
 The SuperColumn implements the IColumn interface.  So you would probably need both the super
column name and the subcolumn name to get at it, but you just need to cast the IColumn to
a super column and handle it from there.
> On Oct 19, 2010, at 10:31 AM, Frank LoVecchio wrote:
>> I have a Hadoop installation working with a cluster of 0.7 Beta 2 Nodes, and got
the WordCount example to work using the standard configuration.  I have been inserting data
into a Super Column (Sensor) with TimeUUID as the compare type, it looks like this:
>> get Sensor['DeviceID:Sensor']
>> => (super_column=795a4da0-d8ac-11df-9a2c-12313d06187c,
>>     (column=sub_sensor1, value=39.742538, timestamp=1287182112633000) 
>>     (column=sub_sensor2, value=-104.912474, timestamp=1287182112633000) 
>>     (column=mac_address, value=DEADBEEFFEED, timestamp=1287182112633000)) 
>> Is there a Word Count example for super columns?  I am trying to count the number
of occurrences of "DEADBEEFFEED", much like "word1" in the column example.  
>> Thanks,
>> Frank LoVecchio
>> Software Engineer, Isidorey LLC

View raw message