incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Freeman <8fo...@gmail.com>
Subject Re: heavy insert load overloads CPUs, with MutationStage pending
Date Fri, 13 Sep 2013 15:52:24 GMT
Paul-  Sorry to go off-list but I'm diving pretty far into details 
here.  Ignore if you wish.

Thanks a lot for the example, definitely very helpful.  I'm surprised 
that the Cassandra experts aren't more interested-in/alarmed-by our 
results, it seems like we've proved that insert performance for wide 
rows in CQL is enormously worse than it was before CQL.  And I have a 
feeling 2.0 won't help much -- I'm already using entirely-prepared batches.

To reproduce your example, I switched to cassandra 1.2.6  and astyanax 
1.56.42.  But anything I try to do with that version combination gives 
me an exception on the client side (e.g. execute() on a query):
> 13-09-13 15:42:42.511 [pool-6-thread-1] ERROR 
> c.n.a.t.ThriftSyncConnectionFactoryImpl - Error creating connection
> java.lang.NoSuchMethodError: 
> org.apache.cassandra.thrift.TBinaryProtocol: method 
> <init>(Lorg/apache/thrift/transport/TTransport;)V not found
>     at 
> com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.open(ThriftSyncConnectionFactoryImpl.java:195)

> ~[astyanax-thrift-1.56.37.jar:na]
>     at 
> com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection$1.run(ThriftSyncConnectionFactoryImpl.java:232)

> [astyanax-thrift-1.56.37.jar:na]
>     at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_07]
 From my googling this is due to a cassandra API change in 
TBinaryProtocol, which is why I had to use cassandra 1.2.5 jars to get 
my astyanax client to work at all in my earlier experiments. Did you 
encounter this?  Also, you had 1.2.8 in the stackoverflow post, but 
1.2.6 in this email, did you have to rollback?

Thanks for any help you can offer, hope I can return the favor at some 
point.


On 09/12/2013 02:26 PM, Paul Cichonski wrote:
> I'm running Cassandra 1.2.6 without compact storage on my tables. The trick is making
your Astyanax (I'm running 1.56.42) mutation work with the CQL table definition (this is definitely
a bit of a hack since most of the advice says don't mix the CQL and Thrift APIs so it is your
call on how far you want to go). If you want to still try and test it out you need to leverage
the Astyanax CompositeColumn construct to make it work (https://github.com/Netflix/astyanax/wiki/Composite-columns)
>
> I've provided a slightly modified version of what I am doing below:
>
> CQL table def:
>
> CREATE TABLE standard_subscription_index
> (
>   	subscription_type text,
> 	subscription_target_id text,
> 	entitytype text,
> 	entityid int,
> 	creationtimestamp timestamp,
> 	indexed_tenant_id uuid,
> 	deleted boolean,
>      PRIMARY KEY ((subscription_type, subscription_target_id), entitytype, entityid)
> )
>
> ColumnFamily definition:
>
> private static final ColumnFamily<SubscriptionIndexCompositeKey, SubscribingEntityCompositeColumn>
COMPOSITE_ROW_COLUMN = new ColumnFamily<SubscriptionIndexCompositeKey, SubscribingEntityCompositeColumn>(
> 	SUBSCRIPTION_CF_NAME, new AnnotatedCompositeSerializer<SubscriptionIndexCompositeKey>(SubscriptionIndexCompositeKey.class),
> 	new AnnotatedCompositeSerializer<SubscribingEntityCompositeColumn>(SubscribingEntityCompositeColumn.class));
>
>
> SubscriptionIndexCompositeKey is a class that contains the fields from the row key (e.g.,
subscription_type, subscription_target_id), and SubscribingEntityCompositeColumn contains
the fields from the composite column (as it would look if you view your data using Cassandra-cli),
so: entityType, entityId, columnName. The columnName field is the tricky part as it defines
what to interpret the column value as (i.e., if it is a value for the creationtimestamp the
column might be "someEntityType:4:creationtimestamp"
>
> The actual mutation looks something like this:
>
> final MutationBatch mutation = getKeyspace().prepareMutationBatch();
> final ColumnListMutation<SubscribingEntityCompositeColumn> row = mutation.withRow(COMPOSITE_ROW_COLUMN,
> 		new SubscriptionIndexCompositeKey(targetEntityType.getName(), targetEntityId));
>
> for (Subscription sub : subs) {
> 	row.putColumn(new SubscribingEntityCompositeColumn(sub.getEntityType().getName(), sub.getEntityId(),
> 				"creationtimestamp"), sub.getCreationTimestamp());
> 	row.putColumn(new SubscribingEntityCompositeColumn(sub.getEntityType().getName(), sub.getEntityId(),
> 				"deleted"), sub.isDeleted());
> 	row.putColumn(new SubscribingEntityCompositeColumn(sub.getEntityType().getName(), sub.getEntityId(),
> 				"indexed_tenant_id"), tenantId);
> }
>
> Hope that helps,
> Paul
>

Mime
View raw message