cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Yudovin <vla...@winguzone.com>
Subject RE: Cassandra Config as per server hardware for heavy write
Date Wed, 23 Nov 2016 08:29:35 GMT
&gt;I have 1Cr records in my Java ArrayList and yes I am writing in sync mode.

Is your Java program single threaded?



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting, Zero production time






---- On Wed, 23 Nov 2016 03:09:29 -0500Abhishek Kumar Maheshwari &lt;Abhishek.Maheshwari@timesinternet.in&gt;
wrote ----




Hi Benjamin,

 

I have 1Cr records in my Java ArrayList and yes I am writing in sync mode. My table is as
below:

 

CREATE TABLE XXX_YY_MMS (

    date timestamp,

    userid text,

    time timestamp,

    xid text,

    addimid text,

    advcid bigint,

    algo bigint,

    alla text,

    aud text,

    bmid text,

    ctyid text,

    bid double,

    ctxid text,

    devipid text,

    gmid text,

    ip text,

    itcid bigint,

    iid text,

    metid bigint,

    osdid text,

    paid int,

    position text,

    pcid bigint,

    refurl text,

    sec text,

    siid bigint,

    tmpid bigint,

    xforwardedfor text,

    PRIMARY KEY (date, userid, time, xid)

) WITH CLUSTERING ORDER BY (userid ASC, time ASC, xid ASC)

    AND bloom_filter_fp_chance = 0.01

    AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'

    AND comment = ''

    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}

    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}

    AND dclocal_read_repair_chance = 0.1

    AND default_time_to_live = 0

    AND gc_grace_seconds = 864000

    AND max_index_interval = 2048

    AND memtable_flush_period_in_ms = 0

    AND min_index_interval = 128

    AND read_repair_chance = 0.0

    AND speculative_retry = '99.0PERCENTILE';

 

So please let me know what I miss?

 

And for this hardware below config is fine?

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

thanks,

Abhishek

 

From: Benjamin Roth [mailto:benjamin.roth@jaumo.com] 
 Sent: Wednesday, November 23, 2016 12:56 PM
 To: user@cassandra.apache.org
 Subject: Re: Cassandra Config as per server hardware for heavy write
 

This is ridiculously slow for that hardware setup. Sounds like you benchmark with a single
thread and / or sync queries or very large writes.

A setup like this should be easily able to handle tens of thousands of writes / s



 

2016-11-23 8:02 GMT+01:00 Jonathan Haddad &lt;jon@jonhaddad.com&gt;:

How are you benchmarking that?

On Tue, Nov 22, 2016 at 9:16 PM Abhishek Kumar Maheshwari &lt;Abhishek.Maheshwari@timesinternet.in&gt;
wrote:


Hi,

 

I have 8 servers in my Cassandra Cluster. Each server has 64 GB ram and 40 Cores and 8 SSD.
Currently I have below config in Cassandra.yaml:

 

concurrent_reads: 32

concurrent_writes: 64

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 32

concurrent_compactors: 8

 

With this configuration, I can write 1700 Request/Sec per server.

 

But our desired write performance is 3000-4000 Request/Sec per server. As per my Understanding
Max value for these parameters can be as below:

concurrent_reads: 32

concurrent_writes: 128(8*16 Corew)

concurrent_counter_writes: 32

compaction_throughput_mb_per_sec: 128

concurrent_compactors: 8 or 16 (as I have 8 SSD and 16 core reserve for this)

 

Please let me know this is fine or I need to tune some other parameters for speedup write.

 

 

Thanks &amp; Regards,
 Abhishek Kumar Maheshwari
 +91- 9999805591 (Mobile)
Times Internet Ltd. | A Times of India Group Company

FC - 6, Sector 16A, Film City,  Noida,  U.P. 201301 | INDIA

P Please do not print this email unless it is absolutely necessary. Spread environmental awareness.

 


Education gets Exciting with IIM Kozhikode Executive Post Graduate Programme in Management
- 2 years (AMBA accredited with full benefits of IIMK Alumni status). Brought to you by IIMK
in association with TSW, an Executive Education initiative from The Times of India Group.
Learn more:  www.timestsw.com












 


--



Benjamin Roth

Prokurist



Jaumo GmbH · www.jaumo.com

Wehrstraße 46 · 73035 Göppingen · Germany

Phone +49 7161 304880-6 · Fax +49 7161 304880-1

AG Ulm · HRB 731058 · Managing Director: Jens Kammerer













Mime
View raw message