incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hodges <j...@somethingsimilar.com>
Subject Re: Casandra insert very slow if i use connection pool
Date Fri, 27 Nov 2009 17:38:04 GMT
Hm, maybe I'm missing something about lazyboy, but it looks like the
port in "localhost:9090" is wrong.
--
Jeff

On Fri, Nov 27, 2009 at 12:55 AM, ruslan usifov <ruslan.usifov@gmail.com> wrote:
> Hello
>
>
> I use follow python program to insert records into cassandra cluster (i my
> test case this is two computers). And i have very very slow inserts (1.4 sec
> per 100 records).
> lazyboy python lib use round robin to chose which connection need use to
> insert.
>
> from lazyboy import *
> from lazyboy.key import Key
>
> import time;
>
> # Define your cluster(s)
> begin = 0;
> btime = time.time();
> connection.add_pool('Keyspace1', ['localhost:9090', '192.168.0.220:9160'])
>
> for i in xrange(begin, begin + 500000):
>   if i!=begin and (i % 10000) == 0:
>     print time.time() - btime;
>     btime = time.time();
>
>   rc = record.Record({'value': i, 'text': "ruslan text"})
>   rc.key = Key("Keyspace1", "Standard1", str(i));
>   rc.save(ZERO);
>
> print time.time() - btime;
>
>
> If i remove from pool 1 connection, all fine. What i do wrong?
>
>
> Here is my test one node configuration. Second gave identical except ip
> addreses
>
>   <ClusterName>Test Cluster</ClusterName>
>
>   <Keyspaces>
>     <Keyspace Name="Keyspace1">
>       <ColumnFamily CompareWith="BytesType"
>                     Name="Standard1"
>                     FlushPeriodInMinutes="60"/>
>       <ColumnFamily CompareWith="UTF8Type" Name="Standard2"/>
>       <ColumnFamily CompareWith="TimeUUIDType" Name="StandardByUUID1"/>
>       <ColumnFamily ColumnType="Super"
>                     CompareWith="UTF8Type"
>                     CompareSubcolumnsWith="UTF8Type"
>                     Name="Super1"/>
>     </Keyspace>
>   </Keyspaces>
>
>   <Partitioner>org.apache.cassandra.dht.RandomPartitioner</Partitioner>
>
>   <InitialToken></InitialToken>
>
>
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>
>
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>
>   <ReplicationFactor>1</ReplicationFactor>
>
>   <CommitLogDirectory>/data/cassandra/commitlog</CommitLogDirectory>
>   <DataFileDirectories>
>       <DataFileDirectory>/data/cassandra/data</DataFileDirectory>
>   </DataFileDirectories>
>   <CalloutLocation>/data/cassandra/callouts</CalloutLocation>
>   <BootstrapFileDirectory>/data/cassandra/bootstrap</BootstrapFileDirectory>
>   <StagingFileDirectory>/data/cassandra/staging</StagingFileDirectory>
>
>   <Seeds>
>       <Seed>192.168.0.37</Seed>
>   </Seeds>
>
>
>   <RpcTimeoutInMillis>5000</RpcTimeoutInMillis>
>   <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>
>
>   <ListenAddress>192.168.0.220</ListenAddress>
>   <StoragePort>7000</StoragePort>
>   <ControlPort>7001</ControlPort>
>
>   <ThriftAddress>0.0.0.0</ThriftAddress>
>   <ThriftPort>9160</ThriftPort>
>   <ThriftFramedTransport>false</ThriftFramedTransport>
>
>
>   <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>
>   <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>   <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>
>   <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>
>   <MemtableSizeInMB>64</MemtableSizeInMB>
>   <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>
>   <ConcurrentReads>8</ConcurrentReads>
>   <ConcurrentWrites>32</ConcurrentWrites>
>
>   <CommitLogSync>periodic</CommitLogSync>
>   <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>   <GCGraceSeconds>864000</GCGraceSeconds>
>
>   <FlushMinThreads>1</FlushMinThreads>
>   <FlushMaxThreads>1</FlushMaxThreads>
>
>   <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>
> </Storage>
>
>

Mime
View raw message