incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Coverston <ben.covers...@datastax.com>
Subject Re: outOfMemory error
Date Thu, 29 Nov 2012 23:04:17 GMT
Damien,

It's a bad idea to ask for very large chunks of data from Cassandra in a
single operation. If you want to get 500MB of data from the server I would
suggest breaking up the requests into smaller chunks (3MB, or smaller) and
page through the data from the client side as you consume it.

If you as for 500MB of data in a single request you'll allocate a 500MB
object on the heap for each request which if done repeatedly, and by
several clients will kill nearly any size heap through fragmentation and GC
pressure.

Ben


On Thu, Nov 29, 2012 at 3:07 AM, Damien Lejeune <d.lejeune@pepite.be> wrote:

> Hi Bryan,
>
> Thanks for the advices, that's indeed more reasonable to proceed with
> predicates and range slices. I was just curious about the Cassandra
> limitation on my machine. But just out of curiosity, do you have an idea
> about the theoretical minimal amount of RAM needed to the Cassandra server
> to fetch 500MB of data at once (if we assume that Cassandra is well
> configured for this test case and if we do not take into account other
> possible troubles generated by the hardware) ? Is there even a rule of
> thumb to calculate that ?
>
> Cheers,
>
> Damien
>
>
> 2012/11/28 Bryan Talbot <btalbot@aeriagames.com>
>
>> Well, asking for 500MB of data at once for a server with such modest
>> specs is asking for troubles.  Here are my suggestions.
>>
>> Disable the 1 GB row cache
>> Consider allocating that memory for the java heap "Xms2500m Xmx2500m"
>> Don't fetch all the columns at once -- page through them a slice at a time
>> Increase the memtable to more than 64 MB if you want to write data to
>> this cluster
>>
>> -Bryan
>>
>>
>>
>> On Wed, Nov 28, 2012 at 5:06 AM, Damien Lejeune <d.lejeune@pepite.be>
>> wrote:
>> > Hi all,
>> >
>> > I'm currently experiencing a outOfMemory problem with Cassandra-1.1.6 on
>> > Windows XP-Pro (32-bit). The server crashes when I try to query it with
>> a
>> > relatively small amount of data (around 100 rows with 50000 columns
>> each: to
>> > be precise, on my configuration, querying 75 or more rows makes the
>> server
>> > to crash).
>> > I tried with different library (Hector, JDBC, Thrift) and with the
>> Cassandra
>> > stress tool. All lead to the same outOfMemory problem.
>> >
>> > My dataset is composed, for each row, of: 1 column in DateType, 49999
>> > columns in DoubleType. I ran a query to fetch the entire dataset (around
>> > 330MB for the raw data + around 200MB for the metadata) and got the log
>> at
>> > the end of this message.
>> >
>> > I also checked the heap-dump with Mat which displays these top values:
>> > Class Name
>> >              Objects                      Shallow Heap
>> > java.nio.HeapByteBuffer                       16,253,559
>> > 780,170,832
>> > bytes[]                                               16,254,013
>> > 330,207,640     <-- Data ?
>> > java.util.TreeMap$Entry                        8,126,711
>> > 260,054,752
>> > org.apache.cassandra.db.Column         8,116,589
>> > 194,798,136     <-- Metadata ?
>> >
>> > I tried to change the configuration in Cassandra for the values:
>> > - row_cache_size_in_mb: tried different value between [0,1000] MB
>> > - flush_largest_memtables_at: set to 0.1, but tried with 0.75
>> > - reduce_cache_sizes_at: tried 0.85, 0.6, 0.2 and 0.1
>> > - reduce_cache_capacity_to: tried 0.6 and 0.15
>> > - memtable_total_space_in_mb: 64 MB, but also tried to disable it (->
>> 1/3 of
>> > the heap)
>> > - Xms1G
>> > - Xmx1500M
>> > with no real observable improvements regarding my problem.
>> >
>> > My Cassandra server and client both run on the same machine.
>> >
>> > Here are the characteristics of my system configuration:
>> > - Cassandra-1.1.6
>> > - java version "1.6.0_20"
>> >          Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
>> >          Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode,
>> sharing)
>> > - Windows XP-Pro 32 bits with service pack 3
>> > - CPU double-core, 32 bits @2.26GHz
>> > - 3.48 of RAM
>> >
>> > I'm aware that my system configuration is not an optimized environment
>> to
>> > make Cassandra to run efficiently, but I wonder if you guys know a
>> > workaround (or any idea on how) to fix this problem. Part of the answer
>> is
>> > probably that I do not have enough RAM to run the process, but I also
>> wonder
>> > if it is a 'normal' behaviour for Cassandra to handle this particular
>> test
>> > case that way.
>> >
>> > Cheers,
>> >
>> > Damien
>> >
>> > ---- Cassandra's LOG ---
>> >
>> > Starting Cassandra Server
>> >  INFO 09:10:27,171 Logging initialized
>> >  INFO 09:10:27,171 JVM vendor/version: Java HotSpot(TM) Client
>> VM/1.6.0_18
>> >  INFO 09:10:27,171 Heap size: 1072103424/1569521664
>> >  INFO 09:10:27,171 Classpath:
>> >
>> E:\dl_benchmark\db\apache-cassandra-1.1.6\conf;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\antlr-3.2.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\apache-cassandra-1.1.6.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\apache-cassandra-clientutil-1.1.6.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\apache-cassandra-thrift-1.1.6.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\avro-1.4.0-fixes.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\avro-1.4.0-sources-fixes.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\commons-cli-1.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\commons-codec-1.2.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\commons-lang-2.4.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\compress-lzf-0.8.4.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\concurrentlinkedhashmap-lru-1.3.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\guava-r08.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\high-scale-lib-1.1.2.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\jackson-core-asl-1.9.2.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\jackson-mapper-asl-1.9.2.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\jamm-0.2.5.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\jline-0.9.94.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\json-simple-1.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\libthrift-0.7.0.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\log4j-1.2.16.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\metrics-core-2.0.3.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\servlet-api-2.5-20081211.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\slf4j-api-1.6.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\slf4j-log4j12-1.6.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\snakeyaml-1.6.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\snappy-java-1.0.4.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\snaptree-0.1.jar;E:\dl_benchmark\db\apache-cassandra-1.1.6\build\classes\main;E:\dl_benchmark\db\apache-cassandra-1.1.6\build\classes\thrift;E:\dl_benchmark\db\apache-cassandra-1.1.6\lib\jamm-0.2.5.jar
>> >  INFO 09:10:27,171 JNA not found. Native methods will be disabled.
>> >  INFO 09:10:27,187 Loading settings from
>> > file:/E:/dl_benchmark/db/apache-cassandra-1.1.6/conf/cassandra.yaml
>> >  INFO 09:10:27,296 32bit JVM detected.  It is recommended to run
>> Cassandra
>> > on a 64bit JVM for better performance.
>> >  INFO 09:10:27,296 DiskAccessMode 'auto' determined to be standard,
>> > indexAccessMode is standard
>> >  INFO 09:10:27,500 Global memtable threshold is enabled at 64MB
>> >  INFO 09:10:27,781 Initializing key cache with capacity of 51 MBs.
>> >  INFO 09:10:27,812 Scheduling key cache save to each 14400 seconds
>> (going to
>> > save all keys).
>> >  INFO 09:10:27,812 Initializing row cache with capacity of 1000 MBs and
>> > provider org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:10:27,812 Scheduling row cache save to each 0 seconds (going to
>> > save all keys).
>> >  INFO 09:10:27,906 reading saved cache
>> > \var\lib\cassandra\saved_caches\system-schema_columnfamilies-KeyCache
>> >  INFO 09:10:27,921 Opening
>> >
>> \var\lib\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-hf-134
>> > (4938 bytes)
>> >  INFO 09:10:27,921 Opening
>> >
>> \var\lib\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-hf-133
>> > (5122 bytes)
>> >  INFO 09:10:27,937 reading saved cache
>> > \var\lib\cassandra\saved_caches\system-schema_columns-KeyCache
>> >  INFO 09:10:27,937 Opening
>> >
>> \var\lib\cassandra\data\system\schema_columns\system-schema_columns-hf-74
>> > (59 bytes)
>> >  INFO 09:10:27,937 Opening
>> >
>> \var\lib\cassandra\data\system\schema_columns\system-schema_columns-hf-73
>> > (13836053 bytes)
>> >  INFO 09:10:27,953 Opening
>> > \var\lib\cassandra\data\system\Versions\system-Versions-hf-64 (247
>> bytes)
>> >  INFO 09:10:27,953 Opening
>> >
>> \var\lib\cassandra\data\system\schema_keyspaces\system-schema_keyspaces-hf-105
>> > (1244 bytes)
>> >  INFO 09:10:27,953 reading saved cache
>> > \var\lib\cassandra\saved_caches\system-LocationInfo-KeyCache
>> >  INFO 09:10:27,953 Opening
>> > \var\lib\cassandra\data\system\LocationInfo\system-LocationInfo-hf-134
>> (346
>> > bytes)
>> >  INFO 09:10:32,843 reading saved cache
>> > \var\lib\cassandra\saved_caches\Keyspace1-Standard1-KeyCache
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-552
>> > (83124150 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-44
>> > (5541610 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-592
>> > (5541610 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-591
>> > (5541610 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-555
>> > (138540250 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-565
>> > (60957710 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-484
>> > (138540250 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-586
>> > (49874490 bytes)
>> >  INFO 09:10:32,859 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-437
>> > (138540250 bytes)
>> >  INFO 09:10:32,875 Opening
>> > \var\lib\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-hf-590
>> > (60957710 bytes)
>> >  INFO 09:10:32,875 Opening
>> > \var\lib\cassandra\data\TESTBENCHTWO\Row\TESTBENCHTWO-Row-hf-133
>> (5514794
>> > bytes)
>> >  INFO 09:10:32,890 Opening
>> > \var\lib\cassandra\data\TESTBENCHTWO\Row\TESTBENCHTWO-Row-hf-130
>> (195755351
>> > bytes)
>> >  INFO 09:10:32,906 Opening
>> > \var\lib\cassandra\data\TESTBENCHTWO\Row\TESTBENCHTWO-Row-hf-131
>> (201295520
>> > bytes)
>> >  INFO 09:10:32,906 Opening
>> > \var\lib\cassandra\data\TESTBENCHTWO\Row\TESTBENCHTWO-Row-hf-132
>> (30327963
>> > bytes)
>> >  INFO 09:10:32,906 Opening
>> > \var\lib\cassandra\data\TESTBENCHTWO\Row\TESTBENCHTWO-Row-hf-129
>> (336374894
>> > bytes)
>> >  INFO 09:10:32,953 Opening
>> > \var\lib\cassandra\data\TESTBENCH\Row\TESTBENCH-Row-hf-198 (91008900
>> bytes)
>> >  INFO 09:10:32,968 Opening
>> > \var\lib\cassandra\data\TESTBENCH\Row\TESTBENCH-Row-hf-202 (57927906
>> bytes)
>> >  INFO 09:10:32,968 Opening
>> > \var\lib\cassandra\data\TESTBENCH\Row\TESTBENCH-Row-hf-188 (2917363977
>> > bytes)
>> >  INFO 09:10:33,156 Opening
>> \var\lib\cassandra\data\DEMO\Row\DEMO-Row-hf-1
>> > (100 bytes)
>> >  INFO 09:10:33,171 Opening
>> > \var\lib\cassandra\data\TESTBENCHTHREE\Row\TESTBENCHTHREE-Row-hf-60
>> > (96519635 bytes)
>> >  INFO 09:10:33,171 Opening
>> > \var\lib\cassandra\data\TESTBENCHTHREE\Row\TESTBENCHTHREE-Row-hf-72
>> > (63929481 bytes)
>> >  INFO 09:10:43,890 completed pre-loading (10 keys) key cache.
>> >  INFO 09:10:44,000 Replaying
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021204.log,
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021205.log
>> >  INFO 09:10:44,015 Replaying
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021204.log
>> >  INFO 09:10:44,031 Finished reading
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021204.log
>> >  INFO 09:10:44,031 Replaying
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021205.log
>> >  INFO 09:10:44,031 Finished reading
>> > \var\lib\cassandra\commitlog\CommitLog-1354090021205.log
>> >  INFO 09:10:44,046 Enqueuing flush of Memtable-Versions@6172767(83/103
>> > serialized/live bytes, 3 ops)
>> >  INFO 09:10:44,062 Writing Memtable-Versions@6172767(83/103
>> serialized/live
>> > bytes, 3 ops)
>> >  INFO 09:10:44,296 Completed flushing
>> > \var\lib\cassandra\data\system\Versions\system-Versions-hf-65-Data.db
>> (247
>> > bytes) for commitlog position ReplayPosition(segmentId=1354090244001,
>> > position=0)
>> >  INFO 09:10:44,312 Log replay complete, 3 replayed mutations
>> >  INFO 09:10:44,890 GC for ConcurrentMarkSweep: 210 ms for 1 collections,
>> > 41139480 used; max is 1569521664
>> >  INFO 09:10:49,906 Cassandra version: 1.1.6
>> >  INFO 09:10:49,906 Thrift API version: 19.32.0
>> >  INFO 09:10:49,906 CQL supported versions: 2.0.0,3.0.0-beta1 (default:
>> > 2.0.0)
>> >  INFO 09:10:49,953 Loading persisted ring state
>> >  INFO 09:10:49,953 Starting up server gossip
>> >  INFO 09:10:49,953 Enqueuing flush of Memtable-LocationInfo@8678192
>> (29/36
>> > serialized/live bytes, 1 ops)
>> >  INFO 09:10:49,968 Writing Memtable-LocationInfo@8678192(29/36
>> > serialized/live bytes, 1 ops)
>> >  INFO 09:10:49,968 Completed flushing
>> >
>> \var\lib\cassandra\data\system\LocationInfo\system-LocationInfo-hf-135-Data.db
>> > (80 bytes) for commitlog position
>> ReplayPosition(segmentId=1354090244001,
>> > position=363)
>> >  INFO 09:10:55,890 Starting Messaging Service on port 7000
>> >  INFO 09:10:55,890 Using saved token
>> 6783448253301637730960311964445940653
>> >  INFO 09:10:55,906 Enqueuing flush of Memtable-LocationInfo@24013134
>> (53/66
>> > serialized/live bytes, 2 ops)
>> >  INFO 09:10:55,906 Writing Memtable-LocationInfo@24013134(53/66
>> > serialized/live bytes, 2 ops)
>> >  INFO 09:10:55,906 Completed flushing
>> >
>> \var\lib\cassandra\data\system\LocationInfo\system-LocationInfo-hf-136-Data.db
>> > (163 bytes) for commitlog position
>> ReplayPosition(segmentId=1354090244001,
>> > position=544)
>> >  INFO 09:10:55,906 Node localhost/127.0.0.1 state jump to normal
>> >  INFO 09:10:55,921 Bootstrap/Replace/Move completed! Now serving reads.
>> >  INFO 09:10:55,953 Binding thrift service to localhost/127.0.0.1:9160
>> >  INFO 09:10:55,953 Using TFastFramedTransport with a max frame size of
>> > 33554432 bytes.
>> >  INFO 09:10:55,953 Using synchronous/threadpool thrift server on
>> > localhost/127.0.0.1 : 9160
>> >  INFO 09:10:55,953 Listening for thrift clients...
>> >  --- The Querying starts here  ---
>> >  INFO 09:14:33,375 GC for ParNew: 742 ms for 1 collections, 1529127672
>> used;
>> > max is 1569521664
>> >  INFO 09:14:33,375 GC for ConcurrentMarkSweep: 7691 ms for 1
>> collections,
>> > 1529135040 used; max is 1569521664
>> >  INFO 09:14:33,421 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:14:33,421 ReadStage                         0         0
>>   0
>> >  INFO 09:14:33,421 RequestResponseStage              0         0
>>   0
>> >  INFO 09:14:33,421 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:14:33,421 MutationStage                     0         0
>>   0
>> >  INFO 09:14:33,421 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:14:33,421 GossipStage                       0         0
>>   0
>> >  INFO 09:14:33,421 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:14:33,421 MigrationStage                    0         0
>>   0
>> >  INFO 09:14:33,421 StreamStage                       0         0
>>   0
>> >  INFO 09:14:33,421 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:14:33,421 FlushWriter                       0         0
>>   0
>> >  INFO 09:14:33,421 MiscStage                         0         0
>>   0
>> >  INFO 09:14:33,421 commitlog_archiver                0         0
>>   0
>> >  INFO 09:14:33,421 InternalResponseStage             0         0
>>   0
>> >  INFO 09:14:33,421 HintedHandoff                     0         0
>>   0
>> >  INFO 09:14:33,421 CompactionManager                 0         0
>> >  INFO 09:14:33,468 MessagingService                n/a       0,0
>> >  INFO 09:14:33,468 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:14:33,468 KeyCache                         14
>> > 1114112                      all
>> >  INFO 09:14:33,468 RowCache                          0
>> > 1048576000                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:14:33,468 ColumnFamily                Memtable ops,data
>> >  INFO 09:14:33,468 system.NodeIdInfo                         0,0
>> >  INFO 09:14:33,468 system.IndexInfo                          0,0
>> >  INFO 09:14:33,468 system.LocationInfo                       0,0
>> >  INFO 09:14:33,468 system.Versions                         3,103
>> >  INFO 09:14:33,468 system.schema_keyspaces                   0,0
>> >  INFO 09:14:33,468 system.Migrations                         0,0
>> >  INFO 09:14:33,468 system.schema_columnfamilies                 0,0
>> >  INFO 09:14:33,468 system.schema_columns                     0,0
>> >  INFO 09:14:33,468 system.HintsColumnFamily                  0,0
>> >  INFO 09:14:33,468 system.Schema                             0,0
>> >  INFO 09:14:33,468 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:14:33,468 Keyspace1.Standard1                       0,0
>> >  INFO 09:14:33,468 Keyspace1.Super1                          0,0
>> >  INFO 09:14:33,468 Keyspace1.Counter1                        0,0
>> >  INFO 09:14:33,468 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:14:33,468 TESTBENCH.Row                             0,0
>> >  INFO 09:14:33,468 DEMO.Row                                  0,0
>> >  INFO 09:14:33,468 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:14:33,468 Heap is 0.9742681958928348 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra is now reducing cache sizes to
>> free
>> > up memory.  Adjust reduce_cache_sizes_at threshold in cassandra.yaml if
>> you
>> > don't want Cassandra to do this automatically
>> >  WARN 09:14:33,468 Reducing RowCache capacity from 1048576000 to 0 to
>> reduce
>> > memory pressure
>> >  WARN 09:14:33,468 Reducing KeyCache capacity from 1114112 to 8 to
>> reduce
>> > memory pressure
>> >  WARN 09:14:33,468 Heap is 0.9742681958928348 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  WARN 09:14:33,468 Flushing CFS(Keyspace='system',
>> ColumnFamily='Versions')
>> > to relieve memory pressure
>> >  INFO 09:14:33,468 Enqueuing flush of Memtable-Versions@9422607(83/103
>> > serialized/live bytes, 3 ops)
>> >  INFO 09:14:33,468 Writing Memtable-Versions@9422607(83/103
>> serialized/live
>> > bytes, 3 ops)
>> >  INFO 09:14:33,562 Completed flushing
>> > \var\lib\cassandra\data\system\Versions\system-Versions-hf-66-Data.db
>> (247
>> > bytes) for commitlog position ReplayPosition(segmentId=1354090244001,
>> > position=544)
>> >  INFO 09:14:42,968 GC for ConcurrentMarkSweep: 8900 ms for 1
>> collections,
>> > 1549381648 used; max is 1569521664
>> >  INFO 09:14:42,968 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:14:42,968 ReadStage                         0         0
>>   0
>> >  INFO 09:14:42,968 RequestResponseStage              0         0
>>   0
>> >  INFO 09:14:42,984 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:14:42,984 MutationStage                     0         0
>>   0
>> >  INFO 09:14:42,984 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:14:43,015 GossipStage                       0         0
>>   0
>> >  INFO 09:14:43,015 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:14:43,015 MigrationStage                    0         0
>>   0
>> >  INFO 09:14:43,015 StreamStage                       0         0
>>   0
>> >  INFO 09:14:43,015 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:14:43,015 FlushWriter                       0         0
>>   0
>> >  INFO 09:14:43,015 MiscStage                         0         0
>>   0
>> >  INFO 09:14:43,015 commitlog_archiver                0         0
>>   0
>> >  INFO 09:14:43,015 InternalResponseStage             0         0
>>   0
>> >  INFO 09:14:43,015 HintedHandoff                     0         0
>>   0
>> >  INFO 09:14:43,015 CompactionManager                 0         0
>> >  INFO 09:14:43,015 MessagingService                n/a       0,0
>> >  INFO 09:14:43,015 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:14:43,015 KeyCache                          8
>> > 8                      all
>> >  INFO 09:14:43,015 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:14:43,015 ColumnFamily                Memtable ops,data
>> >  INFO 09:14:43,015 system.NodeIdInfo                         0,0
>> >  INFO 09:14:43,015 system.IndexInfo                          0,0
>> >  INFO 09:14:43,015 system.LocationInfo                       0,0
>> >  INFO 09:14:43,015 system.Versions                           0,0
>> >  INFO 09:14:43,015 system.schema_keyspaces                   0,0
>> >  INFO 09:14:43,015 system.Migrations                         0,0
>> >  INFO 09:14:43,015 system.schema_columnfamilies                 0,0
>> >  INFO 09:14:43,015 system.schema_columns                     0,0
>> >  INFO 09:14:43,015 system.HintsColumnFamily                  0,0
>> >  INFO 09:14:43,015 system.Schema                             0,0
>> >  INFO 09:14:43,015 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:14:43,015 Keyspace1.Standard1                       0,0
>> >  INFO 09:14:43,015 Keyspace1.Super1                          0,0
>> >  INFO 09:14:43,015 Keyspace1.Counter1                        0,0
>> >  INFO 09:14:43,015 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:14:43,015 TESTBENCH.Row                             0,0
>> >  INFO 09:14:43,015 DEMO.Row                                  0,0
>> >  INFO 09:14:43,015 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:14:43,015 Heap is 0.9871680547889525 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:14:43,015 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:14:50,046 GC for ConcurrentMarkSweep: 6530 ms for 1
>> collections,
>> > 1561411240 used; max is 1569521664
>> >  INFO 09:14:50,234 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:14:50,234 ReadStage                         0         0
>>   0
>> >  INFO 09:14:50,234 RequestResponseStage              0         0
>>   0
>> >  INFO 09:14:50,234 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:14:50,234 MutationStage                     0         0
>>   0
>> >  INFO 09:14:50,234 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:14:50,234 GossipStage                       0         0
>>   0
>> >  INFO 09:14:50,234 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:14:50,234 MigrationStage                    0         0
>>   0
>> >  INFO 09:14:50,234 StreamStage                       0         0
>>   0
>> >  INFO 09:14:50,234 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:14:50,234 FlushWriter                       0         0
>>   0
>> >  INFO 09:14:50,234 MiscStage                         0         0
>>   0
>> >  INFO 09:14:50,234 commitlog_archiver                0         0
>>   0
>> >  INFO 09:14:50,234 InternalResponseStage             0         0
>>   0
>> >  INFO 09:14:50,234 HintedHandoff                     0         0
>>   0
>> >  INFO 09:14:50,234 CompactionManager                 0         0
>> >  INFO 09:14:50,234 MessagingService                n/a       0,0
>> >  INFO 09:14:50,234 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:14:50,234 KeyCache                          8
>> > 8                      all
>> >  INFO 09:14:50,234 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:14:50,234 ColumnFamily                Memtable ops,data
>> >  INFO 09:14:50,234 system.NodeIdInfo                         0,0
>> >  INFO 09:14:50,234 system.IndexInfo                          0,0
>> >  INFO 09:14:50,234 system.LocationInfo                       0,0
>> >  INFO 09:14:50,234 system.Versions                           0,0
>> >  INFO 09:14:50,234 system.schema_keyspaces                   0,0
>> >  INFO 09:14:50,234 system.Migrations                         0,0
>> >  INFO 09:14:50,234 system.schema_columnfamilies                 0,0
>> >  INFO 09:14:50,234 system.schema_columns                     0,0
>> >  INFO 09:14:50,265 system.HintsColumnFamily                  0,0
>> >  INFO 09:14:50,265 system.Schema                             0,0
>> >  INFO 09:14:50,265 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:14:50,265 Keyspace1.Standard1                       0,0
>> >  INFO 09:14:50,265 Keyspace1.Super1                          0,0
>> >  INFO 09:14:50,265 Keyspace1.Counter1                        0,0
>> >  INFO 09:14:50,265 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:14:50,265 TESTBENCH.Row                             0,0
>> >  INFO 09:14:50,265 DEMO.Row                                  0,0
>> >  INFO 09:14:50,265 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:14:50,265 Heap is 0.9948325504604185 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:14:50,265 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:14:59,343 GC for ConcurrentMarkSweep: 8724 ms for 1
>> collections,
>> > 1566184304 used; max is 1569521664
>> >  INFO 09:14:59,343 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:14:59,343 ReadStage                         0         0
>>   0
>> >  INFO 09:14:59,343 RequestResponseStage              0         0
>>   0
>> >  INFO 09:14:59,343 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:14:59,343 MutationStage                     0         0
>>   0
>> >  INFO 09:14:59,343 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:14:59,343 GossipStage                       0         0
>>   0
>> >  INFO 09:14:59,343 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:14:59,343 MigrationStage                    0         0
>>   0
>> >  INFO 09:14:59,343 StreamStage                       0         0
>>   0
>> >  INFO 09:14:59,343 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:14:59,343 FlushWriter                       0         0
>>   0
>> >  INFO 09:14:59,343 MiscStage                         0         0
>>   0
>> >  INFO 09:14:59,343 commitlog_archiver                0         0
>>   0
>> >  INFO 09:14:59,343 InternalResponseStage             0         0
>>   0
>> >  INFO 09:14:59,343 HintedHandoff                     0         0
>>   0
>> >  INFO 09:14:59,343 CompactionManager                 0         0
>> >  INFO 09:14:59,343 MessagingService                n/a       0,0
>> >  INFO 09:14:59,343 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:14:59,343 KeyCache                          8
>> > 8                      all
>> >  INFO 09:14:59,343 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:14:59,343 ColumnFamily                Memtable ops,data
>> >  INFO 09:14:59,343 system.NodeIdInfo                         0,0
>> >  INFO 09:14:59,343 system.IndexInfo                          0,0
>> >  INFO 09:14:59,343 system.LocationInfo                       0,0
>> >  INFO 09:14:59,343 system.Versions                           0,0
>> >  INFO 09:14:59,343 system.schema_keyspaces                   0,0
>> >  INFO 09:14:59,343 system.Migrations                         0,0
>> >  INFO 09:14:59,343 system.schema_columnfamilies                 0,0
>> >  INFO 09:14:59,343 system.schema_columns                     0,0
>> >  INFO 09:14:59,343 system.HintsColumnFamily                  0,0
>> >  INFO 09:14:59,343 system.Schema                             0,0
>> >  INFO 09:14:59,343 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:14:59,343 Keyspace1.Standard1                       0,0
>> >  INFO 09:14:59,343 Keyspace1.Super1                          0,0
>> >  INFO 09:14:59,343 Keyspace1.Counter1                        0,0
>> >  INFO 09:14:59,343 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:14:59,343 TESTBENCH.Row                             0,0
>> >  INFO 09:14:59,343 DEMO.Row                                  0,0
>> >  INFO 09:14:59,343 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:14:59,343 Heap is 0.997873645151546 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:14:59,343 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:15:08,500 GC for ConcurrentMarkSweep: 8854 ms for 1
>> collections,
>> > 1567778080 used; max is 1569521664
>> >  INFO 09:15:08,500 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:15:08,500 ReadStage                         0         0
>>   0
>> >  INFO 09:15:08,500 RequestResponseStage              0         0
>>   0
>> >  INFO 09:15:08,500 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:15:08,500 MutationStage                     0         0
>>   0
>> >  INFO 09:15:08,500 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:15:08,500 GossipStage                       0         0
>>   0
>> >  INFO 09:15:08,515 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:15:08,515 MigrationStage                    0         0
>>   0
>> >  INFO 09:15:08,515 StreamStage                       0         0
>>   0
>> >  INFO 09:15:08,515 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:15:08,515 FlushWriter                       0         0
>>   0
>> >  INFO 09:15:08,515 MiscStage                         0         0
>>   0
>> >  INFO 09:15:08,515 commitlog_archiver                0         0
>>   0
>> >  INFO 09:15:08,515 InternalResponseStage             0         0
>>   0
>> >  INFO 09:15:08,515 HintedHandoff                     0         0
>>   0
>> >  INFO 09:15:08,515 CompactionManager                 0         0
>> >  INFO 09:15:08,515 MessagingService                n/a       0,0
>> >  INFO 09:15:08,515 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:15:08,515 KeyCache                          8
>> > 8                      all
>> >  INFO 09:15:08,515 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:15:08,515 ColumnFamily                Memtable ops,data
>> >  INFO 09:15:08,515 system.NodeIdInfo                         0,0
>> >  INFO 09:15:08,515 system.IndexInfo                          0,0
>> >  INFO 09:15:08,515 system.LocationInfo                       0,0
>> >  INFO 09:15:08,515 system.Versions                           0,0
>> >  INFO 09:15:08,515 system.schema_keyspaces                   0,0
>> >  INFO 09:15:08,515 system.Migrations                         0,0
>> >  INFO 09:15:08,515 system.schema_columnfamilies                 0,0
>> >  INFO 09:15:08,515 system.schema_columns                     0,0
>> >  INFO 09:15:08,515 system.HintsColumnFamily                  0,0
>> >  INFO 09:15:08,515 system.Schema                             0,0
>> >  INFO 09:15:08,515 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:15:08,515 Keyspace1.Standard1                       0,0
>> >  INFO 09:15:08,515 Keyspace1.Super1                          0,0
>> >  INFO 09:15:08,515 Keyspace1.Counter1                        0,0
>> >  INFO 09:15:08,515 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:15:08,515 TESTBENCH.Row                             0,0
>> >  INFO 09:15:08,515 DEMO.Row                                  0,0
>> >  INFO 09:15:08,515 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:15:08,515 Heap is 0.9988890984814084 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:15:08,515 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:15:13,734 GC for ConcurrentMarkSweep: 5129 ms for 1
>> collections,
>> > 1568572088 used; max is 1569521664
>> >  INFO 09:15:13,734 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:15:13,734 ReadStage                         0         0
>>   0
>> >  INFO 09:15:13,734 RequestResponseStage              0         0
>>   0
>> >  INFO 09:15:13,734 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:15:13,734 MutationStage                     0         0
>>   0
>> >  INFO 09:15:13,734 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:15:13,734 GossipStage                       0         0
>>   0
>> >  INFO 09:15:13,734 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:15:13,734 MigrationStage                    0         0
>>   0
>> >  INFO 09:15:13,734 StreamStage                       0         0
>>   0
>> >  INFO 09:15:13,734 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:15:13,734 FlushWriter                       0         0
>>   0
>> >  INFO 09:15:13,734 MiscStage                         0         0
>>   0
>> >  INFO 09:15:13,765 commitlog_archiver                0         0
>>   0
>> >  INFO 09:15:13,765 InternalResponseStage             0         0
>>   0
>> >  INFO 09:15:13,765 HintedHandoff                     0         0
>>   0
>> >  INFO 09:15:13,765 CompactionManager                 0         0
>> >  INFO 09:15:13,765 MessagingService                n/a       0,0
>> >  INFO 09:15:13,765 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:15:13,765 KeyCache                          8
>> > 8                      all
>> >  INFO 09:15:13,765 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:15:13,765 ColumnFamily                Memtable ops,data
>> >  INFO 09:15:13,765 system.NodeIdInfo                         0,0
>> >  INFO 09:15:13,765 system.IndexInfo                          0,0
>> >  INFO 09:15:13,765 system.LocationInfo                       0,0
>> >  INFO 09:15:13,765 system.Versions                           0,0
>> >  INFO 09:15:13,765 system.schema_keyspaces                   0,0
>> >  INFO 09:15:13,765 system.Migrations                         0,0
>> >  INFO 09:15:13,765 system.schema_columnfamilies                 0,0
>> >  INFO 09:15:13,765 system.schema_columns                     0,0
>> >  INFO 09:15:13,765 system.HintsColumnFamily                  0,0
>> >  INFO 09:15:13,765 system.Schema                             0,0
>> >  INFO 09:15:13,765 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:15:13,765 Keyspace1.Standard1                       0,0
>> >  INFO 09:15:13,765 Keyspace1.Super1                          0,0
>> >  INFO 09:15:13,765 Keyspace1.Counter1                        0,0
>> >  INFO 09:15:13,765 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:15:13,765 TESTBENCH.Row                             0,0
>> >  INFO 09:15:13,765 DEMO.Row                                  0,0
>> >  INFO 09:15:13,765 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:15:13,765 Heap is 0.999394990192375 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:15:13,765 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:15:21,484 GC for ConcurrentMarkSweep: 7718 ms for 1
>> collections,
>> > 1568953672 used; max is 1569521664
>> >  INFO 09:15:21,484 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:15:21,484 ReadStage                         0         0
>>   0
>> >  INFO 09:15:21,484 RequestResponseStage              0         0
>>   0
>> >  INFO 09:15:21,484 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:15:21,484 MutationStage                     0         0
>>   0
>> >  INFO 09:15:21,484 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:15:21,484 GossipStage                       0         0
>>   0
>> >  INFO 09:15:21,484 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:15:21,500 MigrationStage                    0         0
>>   0
>> >  INFO 09:15:21,500 StreamStage                       0         0
>>   0
>> >  INFO 09:15:21,500 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:15:35,750 FlushWriter                       0         0
>>   0
>> >  INFO 09:15:35,750 MiscStage                         0         0
>>   0
>> >  INFO 09:15:35,750 commitlog_archiver                0         0
>>   0
>> >  INFO 09:15:35,750 InternalResponseStage             0         0
>>   0
>> >  INFO 09:15:35,750 HintedHandoff                     0         0
>>   0
>> >  INFO 09:15:35,750 CompactionManager                 0         0
>> >  INFO 09:15:35,750 MessagingService                n/a       0,0
>> >  INFO 09:15:35,750 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:16:26,046 KeyCache                          8
>> > 8                      all
>> >  INFO 09:16:31,140 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:16:36,296 ColumnFamily                Memtable ops,data
>> >  INFO 09:16:36,296 system.NodeIdInfo                         0,0
>> >  INFO 09:16:36,296 system.IndexInfo                          0,0
>> >  INFO 09:16:41,437 system.LocationInfo                       0,0
>> >  INFO 09:16:41,437 system.Versions                           0,0
>> > java.lang.OutOfMemoryError: Java heap space
>> > Dumping heap to java_pid5316.hprof ...
>> > Heap dump file created [1755520972 bytes in 84.007 secs]
>> >  INFO 09:21:06,562 system.schema_keyspaces                   0,0
>> >  INFO 09:24:11,906 Stop listening to thrift clients
>> >  INFO 09:24:24,343 system.Migrations                         0,0
>> > ERROR 09:24:24,343 Exception in thread Thread[COMMIT-LOG-WRITER,5,main]
>> > java.lang.OutOfMemoryError: Java heap space
>> >     at
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1739)
>> >     at
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1954)
>> >     at
>> >
>> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
>> >     at
>> >
>> org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:46)
>> >     at
>> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>> >     at java.lang.Thread.run(Thread.java:619)
>> > ERROR 09:24:24,359 Exception in thread
>> > Thread[PERIODIC-COMMIT-LOG-SYNCER,5,main]
>> > java.lang.OutOfMemoryError: Java heap space
>> > ERROR 09:24:24,359 Exception in thread Thread[OptionalTasks:1,5,main]
>> > java.lang.OutOfMemoryError: Java heap space
>> > ERROR 09:24:24,359 Exception in thread Thread[RMI TCP
>> Connection(idle),5,RMI
>> > Runtime]
>> > java.lang.OutOfMemoryError: Java heap space
>> >     at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59)
>> >     at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:42)
>> >     at
>> >
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:725)
>> >     at
>> >
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>> >     at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >     at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >     at java.lang.Thread.run(Thread.java:619)
>> > ERROR 09:24:24,359 Exception in thread Thread[Thrift:1,5,main]
>> > java.lang.OutOfMemoryError: Java heap space
>> >  INFO 09:24:24,343 system.schema_columnfamilies                 0,0
>> >  INFO 09:24:24,421 system.schema_columns                     0,0
>> >  INFO 09:24:24,421 system.HintsColumnFamily                  0,0
>> >  INFO 09:24:24,421 system.Schema                             0,0
>> >  INFO 09:24:24,421 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:24:24,421 Keyspace1.Standard1                       0,0
>> >  INFO 09:24:24,421 Keyspace1.Super1                          0,0
>> >  INFO 09:24:24,421 Keyspace1.Counter1                        0,0
>> >  INFO 09:24:24,421 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:24:24,421 TESTBENCH.Row                             0,0
>> >  INFO 09:24:24,421 DEMO.Row                                  0,0
>> >  INFO 09:24:24,421 TESTBENCHTHREE.Row                        0,0
>> >  WARN 09:24:24,421 Heap is 0.9996381113985057 full.  You may need to
>> reduce
>> > memtable and/or cache sizes.  Cassandra will now flush up to the two
>> largest
>> > memtables to free up memory.  Adjust flush_largest_memtables_at
>> threshold in
>> > cassandra.yaml if you don't want Cassandra to do this automatically
>> >  INFO 09:24:24,421 Unable to reduce heap usage since there are no dirty
>> > column families
>> >  INFO 09:24:24,484 Announcing shutdown
>> >  INFO 09:24:25,421 GC for ConcurrentMarkSweep: 463729 ms for 86
>> collections,
>> > 22709920 used; max is 1569521664
>> >  INFO 09:24:25,421 Pool Name                    Active   Pending
>> Blocked
>> >  INFO 09:24:25,421 ReadStage                         0         0
>>   0
>> >  INFO 09:24:25,421 RequestResponseStage              0         0
>>   0
>> >  INFO 09:24:25,421 ReadRepairStage                   0         0
>>   0
>> >  INFO 09:24:25,421 MutationStage                     0         0
>>   0
>> >  INFO 09:24:25,421 ReplicateOnWriteStage             0         0
>>   0
>> >  INFO 09:24:25,421 GossipStage                       0         0
>>   0
>> >  INFO 09:24:25,421 AntiEntropyStage                  0         0
>>   0
>> >  INFO 09:24:25,421 MigrationStage                    0         0
>>   0
>> >  INFO 09:24:25,437 StreamStage                       0         0
>>   0
>> >  INFO 09:24:25,437 MemtablePostFlusher               0         0
>>   0
>> >  INFO 09:24:25,437 FlushWriter                       0         0
>>   0
>> >  INFO 09:24:25,437 MiscStage                         0         0
>>   0
>> >  INFO 09:24:25,437 commitlog_archiver                0         0
>>   0
>> >  INFO 09:24:25,437 InternalResponseStage             0         0
>>   0
>> >  INFO 09:24:25,437 HintedHandoff                     0         0
>>   0
>> >  INFO 09:24:25,437 CompactionManager                 0         0
>> >  INFO 09:24:25,437 MessagingService                n/a       0,0
>> >  INFO 09:24:25,437 Cache Type                     Size
>> > Capacity               KeysToSave
>> > Provider
>> >  INFO 09:24:25,437 KeyCache                          8
>> > 8                      all
>> >  INFO 09:24:25,437 RowCache                          0
>> > 0                      all
>> > org.apache.cassandra.cache.SerializingCacheProvider
>> >  INFO 09:24:25,437 ColumnFamily                Memtable ops,data
>> >  INFO 09:24:25,437 system.NodeIdInfo                         0,0
>> >  INFO 09:24:25,437 system.IndexInfo                          0,0
>> >  INFO 09:24:25,437 system.LocationInfo                       0,0
>> >  INFO 09:24:25,437 system.Versions                           0,0
>> >  INFO 09:24:25,437 system.schema_keyspaces                   0,0
>> >  INFO 09:24:25,437 system.Migrations                         0,0
>> >  INFO 09:24:25,437 system.schema_columnfamilies                 0,0
>> >  INFO 09:24:25,437 system.schema_columns                     0,0
>> >  INFO 09:24:25,437 system.HintsColumnFamily                  0,0
>> >  INFO 09:24:25,437 system.Schema                             0,0
>> >  INFO 09:24:25,437 Keyspace1.SuperCounter1                   0,0
>> >  INFO 09:24:25,437 Keyspace1.Standard1                       0,0
>> >  INFO 09:24:25,437 Keyspace1.Super1                          0,0
>> >  INFO 09:24:25,437 Keyspace1.Counter1                        0,0
>> >  INFO 09:24:25,437 TESTBENCHTWO.Row                          0,0
>> >  INFO 09:24:25,437 TESTBENCH.Row                             0,0
>> >  INFO 09:24:25,437 DEMO.Row                                  0,0
>> >  INFO 09:24:25,437 TESTBENCHTHREE.Row                        0,0
>> >  INFO 09:24:25,484 Waiting for messaging service to quiesce
>> >  INFO 09:24:25,531 MessagingService shutting down server thread.
>> >
>> > ---- End of LOG -----
>>
>
>


-- 
Ben Coverston
DataStax -- The Apache Cassandra Company

Mime
View raw message