cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan <cne...@yahoo.com>
Subject Re: Write timeout under load but Read is fine
Date Thu, 05 Mar 2015 20:51:15 GMT
Hello Jaydeep;
Run cassandra-stress with R/W options enabled  for about the same time and check if you have
dropped packets. It would eliminate the client as the source of the error & also give
you a replicable tool to base subsequent tests/ findings. 
Jan/ 

 

     On Thursday, March 5, 2015 12:19 PM, Jaydeep Chovatia <chovatia.jaydeep@gmail.com>
wrote:
   

 I have tried increasing timeout to 10000 but no help. Also verified that there is no network
lost packets.
Jaydeep
On Wed, Mar 4, 2015 at 12:19 PM, Jan <cnet62@yahoo.com> wrote:

HI Jaydeep; 
   
   - look at the i/o  on all three nodes   

   - Increase the write_request_timeout_in_ms: 10000   

   - check the time-outs if any on the client inserting the Writes   

   - check the Network for  dropped/lost packets   


hope this helpsJan/
 

     On Wednesday, March 4, 2015 12:26 PM, Jaydeep Chovatia <chovatia.jaydeep@gmail.com>
wrote:
   

 Hi,
In my test program when I increase load then I keep getting few "write timeout" from Cassandra
say every 10~15 mins. My read:write ratio is 50:50. My reads are fine but only writes time
out.

Here is my Cassandra details:Version: 2.0.11
Ring of 3 nodes with RF=3Node configuration: 24 core + 64GB RAM + 2TB
"write_request_timeout_in_ms: 5000", rest of Cassandra.yaml configuration is default
I've also checked IO on Cassandra nodes and looks very low (around 5%). I've also checked
Cassandra log file and do not see any GC happening. Also CPU on Cassandra is low (around 20%).
I have 20GB data on each node.
My test program creates connection to all three Cassandra nodes and sends read+write request
randomly. 
Any idea what should I look for?
Jaydeep


    



-- 
Jaydeep

   
Mime
View raw message