cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayank Mishra <>
Subject Re: ColumnFamilyRecordWriter
Date Tue, 01 Mar 2011 14:37:57 GMT
Thanks Jeremy,

It make sense to abstract out CFOF and CFRW (right now it's tightly 
bounded to avro), so that one can plugin custom serializer (avro, thrift 
and going forward I guess may be CQL). I will create a JIRA and submit 
the patch with do the needful changes. Surely, I will ping you if I 
require help.

With regards,

On 28-02-2011 23:07, Jeremy Hanna wrote:
> One thing that could be done is the CFRW could be abstracted more so that it's easier
to extend and only the serialization mechanism is required to extend it.  That is, all of
the core functionality relating to Cassandra would be in an abstract class or something like
that.  Then the avro based one could extend that with things specific to avro.  That way people
could write their own CFRW extension with whatever serialization they chose.  Anyway, that
seems reasonable, but would take some work - if you'd like to look at that, I could help as
I had time.
> On Feb 28, 2011, at 10:19 AM, Jeremy Hanna wrote:
>> There certainly could be a thrift based record writer.  However, (if I remember correctly)
to enable Hadoop output streaming, it was easier to go with Avro for doing the records as
the schema is included.  There could also have been a thrift version of the record writer,
but it's simpler to just have one record writer.  That was the decision process at least.
>> If there is a compelling reason or a lot of demand for a thrift based one, maybe
it could be revisited - though I'm not the one making that decision.
>> On Feb 28, 2011, at 4:10 AM, Mayank Mishra wrote:
>>> Hi all,
>>> As I was integrating Hadoop with Cassandra, I wanted to serialize mutations,
hence I used thrift mutations in M/R jobs.
>>> During the course, I came to know that CFRW considers only Avro mutations. Can
someone please explain me why only avro transport is entertained by CFRW. Why not, both thrift
and avro mutations are considered?
>>> Please let me know if I missed some important point.
>>> With regards,
>>> Mayank

View raw message