cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomas Zulberti <tzulbe...@gmail.com>
Subject Bulk insertion in Cassandra 0.7 beta3
Date Tue, 09 Nov 2010 19:26:36 GMT
We are making some tests using 3 nodes: A, B, C. We are bulk inserting
87500 keys, and for each of them 1 super column with 768 columns.
We are using hector 0.7.0-18 to insert the data, and at some point an
exception is raised, and sometimes cassandra deamon stop running in
one of the nodes.
The nodes has 2gb of RAM, so the JVM heap is 1gb, and the CPU load
goes up to 80%.

It is possible to insert that ammount of data every 10 minutes? That
would be our use case scenario. We are newbies in cassandra, so maybe
we must take a different approach. What do you suggest?

The keyspace configuration is:
    - name: TestKeyspace
      replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy
      replication_factor: 3
      column_families:
        - name: TestFamily
          column_type: Super
          compare_with: LongType
          compare_subcolumns_with: UTF8Type
          keys_cached: 200000
          rows_cached: 0
          key_cache_save_period_in_seconds: 3600
          row_cache_save_period_in_seconds: 0
          memtable_flush_after_mins: 3600
          memtable_throughput_in_mb: 80
          memtable_operations_in_millions: 0.10

the seeds configuration for each node in each machine:
seeds:
    - A

and the exception that is raised is:
Exception in thread "main"
me.prettyprint.hector.api.exceptions.HUnavailableException:
UnavailableException()
        at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:36)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$1.execute(KeyspaceServiceImpl.java:88)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$1.execute(KeyspaceServiceImpl.java:81)
        at me.prettyprint.cassandra.service.Operation.executeAndSetResult(FailoverOperator.java:388)
        at me.prettyprint.cassandra.service.FailoverOperator.operateSingleIteration(FailoverOperator.java:194)
        at me.prettyprint.cassandra.service.FailoverOperator.operate(FailoverOperator.java:99)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:123)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.batchMutate(KeyspaceServiceImpl.java:93)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.batchMutate(KeyspaceServiceImpl.java:99)
        at me.prettyprint.cassandra.model.MutatorImpl$2.doInKeyspace(MutatorImpl.java:142)
        at me.prettyprint.cassandra.model.MutatorImpl$2.doInKeyspace(MutatorImpl.java:139)
        at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
        at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:58)
        at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:139)
        at com.popego.benchmarks.InsertSegmentsHits.batchInsert(InsertSegmentsHits.java:154)
        at com.popego.benchmarks.InsertSegmentsHits.insertData(InsertSegmentsHits.java:131)
        at com.popego.benchmarks.InsertSegmentsHits.main(InsertSegmentsHits.java:177)
Caused by: UnavailableException()
        at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:16633)
        at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:935)
        at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:909)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$1.execute(KeyspaceServiceImpl.java:86)
        ... 15 more

Mime
View raw message