cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wxn002@zjqunshuo.com" <wxn...@zjqunshuo.com>
Subject Is it a memory issue?
Date Mon, 07 Nov 2016 03:17:19 GMT
Hi All,
We have one issue on C* testing. At first the inserting was very fast and TPS was about 30K/s,
but when the size of data rows reached 2 billion, the insertion rate decreased very badly
and the TPS was 20K/s. When the size of rows reached 2.3 billion, the TPS decreased to 0.5K/s,
and writing timeout come out. At last OOM issue happened in some nodes and C* deamon in some
nodes crashed.  In production we have about 8 billion rows. My testing cluster setting is
as below. My question is if the memory is the main issue. Do I need increase the memory, and
what's the right setting for MAX_HEAP_SIZE and HEAP_NEWSIZE?

My cluster setting:
C* cluster with 3 nodes in Aliyun Cloud
CPU: 4core
Memory: 8G
Disk: 500G
MAX_HEAP_SIZE=2G
HEAP_NEWSIZE=500M

My table schema:
CREATE KEYSPACE IF NOT EXISTS cargts WITH REPLICATION = {'class': 'SimpleStrategy','replication_factor':2};
use cargts;
CREATE TABLE eventdata (
deviceId int,
date int,
event_time bigint,
lat decimal,
lon decimal,
speed int,
heading int,
PRIMARY KEY ((deviceId,date),event_time)
)
WITH CLUSTERING ORDER BY (event_time ASC);
CREATE INDEX ON eventdata (event_time);

Best Regards,
-Simon Wu

Mime
View raw message