cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Hanna <jeremy.hanna1...@gmail.com>
Subject Re: [hadoop] Counters in ColumnFamilyOutputFormat?
Date Thu, 19 May 2011 15:52:52 GMT
FWIW, as I mentioned in the 1497 comments, the patch makes it abstract so that you can have
any rpc/marshalling format you want with a simple extension point.  So if we want to move
to something besides avro, or even like I mentioned do something with Dumbo for streaming,
it's easy to extend.

On May 19, 2011, at 9:23 AM, Jonathan Ellis wrote:

> Avro is there because (1) a long time ago (it now seems) we thought we
> were going to move the main RPC layer to Avro and (2) it simplifies
> using Avro for Streaming, which also seems misguided now
> (https://issues.apache.org/jira/browse/CASSANDRA-1497).
> 
> Using "native" Thrift mutations makes the most sense to me now, which
> would keep it similar in structure but avoid the avroToThrift copy.
> 
> On Thu, May 19, 2011 at 2:30 AM, Mck <mck@apache.org> wrote:
>> I'd like to investigate using Counters in hadoop using
>> ColumnFamilyOutputFormat.
>> 
>> But i see that this class uses outdated ..hadoop.arvo classes
>> 
>> Does it make sense to use counters for hadoop output?
>> 
>> If i try rewriting ColumnFamilyOutputFormat and friends should it be to
>> the normal ..avro classes, or to something else?
>> 
>> ~mck
>> 
>> 
> 
> 
> 
> -- 
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com


Mime
View raw message