cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Hanna <jeremy.hanna1...@gmail.com>
Subject Re: ColumnFamilyOutputFormat and mapreduce.output.columnfamilyoutputformat.batch.threshold 0.70beta1
Date Thu, 09 Sep 2010 15:46:58 GMT
When Jonathan said don't build trunk thrift, he meant just thrift - apply the patches against
cassandra trunk.  You shouldn't need to build the thrift bindings.

On Sep 8, 2010, at 10:54 PM, Alex Burkoff wrote:

> Well, 7.0beta1 rejects those patches. Is there a specific revision I can try
> applying them to ?
> 
> Alex.
> ________________________________________
> From: Jonathan Ellis [jbellis@gmail.com]
> Sent: Wednesday, September 08, 2010 6:48 PM
> To: user@cassandra.apache.org
> Subject: Re: ColumnFamilyOutputFormat and mapreduce.output.columnfamilyoutputformat.batch.threshold
0.70beta1
> 
> You can't build Cassandra against trunk thrift, the API has changed.
> Stick to the one shipped w/ Cassandra and you will be fine.
> 
> On Wed, Sep 8, 2010 at 5:41 PM, Alex Burkoff <ABurkoff@clearpathnet.com> wrote:
>> With the trunk version and given patches I am now getting following exception:
>> 
>> 10/09/08 22:39:14 WARN mapred.LocalJobRunner: job_local_0001
>> java.lang.ClassCastException: [B cannot be cast to java.nio.ByteBuffer
>>       at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.write(ColumnFamilyRecordWriter.java:68)
>>       at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:508)
>>       at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>>       at cassandratest.Main$TReducer.reduce(Main.java:132)
>>       at cassandratest.Main$TReducer.reduce(Main.java:113)
>>       at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>>       at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
>>       at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>>       at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
>> 10/09/08 22:39:14 INFO mapred.JobClient:  map 100% reduce 0%
>> 
>> Alex.
>> ________________________________________
>> From: Jonathan Ellis [jbellis@gmail.com]
>> Sent: Wednesday, September 08, 2010 2:26 PM
>> To: user@cassandra.apache.org
>> Subject: Re: ColumnFamilyOutputFormat and mapreduce.output.columnfamilyoutputformat.batch.threshold
0.70beta1
>> 
>> Try the patches on
>> https://issues.apache.org/jira/browse/CASSANDRA-1434 (or wait until
>> they're committed to trunk, then try a nightly build)
>> 
>> On Wed, Sep 8, 2010 at 4:18 PM, Alex Burkoff <ABurkoff@clearpathnet.com> wrote:
>>> Guys,
>>> 
>>> I was testing ColumnFamilyOutputFormat and found that only columns from the last
Reduce
>>> invocation get stored when mapreduce.output.columnfamilyoutputformat.batch.threshold
has
>>> the default value. Setting it to 1 changes the behavior, and all data is stored
then. Is it the
>>> intended behavior, or am I missing something ?
>>> 
>>> Best regards,
>>> 
>>> Alex Burkoff
>>> 
>> 
>> 
>> 
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>> 
> 
> 
> 
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com


Mime
View raw message