NareshThanksThanks Alain, will avoid caps....i am newbie to cassandra, just started using 2 weeks back..
Here are JConsole screenshots just 5mins after cassandra freezed :
http://i.imgur.com/zxhFzr1.png 4:05 is time cassandra freezed thats why decline in no of threads
http://i.imgur.com/ScgAciv.pngUploaded complete system.log of cassandra till freeze : http://www.scribd.com/doc/159949231/Cassandrasystem-log
Observation : As in my usecase i am storing 1lakh combinations(527insert,506update,954select) each parallel by 100 threads in batch of 1000...Sometimes it works till 1000 batch then hangs but sometimes it completes 10000 then hangs and once even worked for more than lakh....
Same hardware Same settings of cassandra i see random behaviour of performance..On Tue, Aug 13, 2013 at 3:48 PM, Alain RODRIGUEZ <firstname.lastname@example.org> wrote:Hi Naresh.First thing, there is no need of caps in here. People reading this ML is here to help when they have time and skills enough to do so. So please, chill out and do not use caps to show how much desperate you are.Concerning your problem, the only abnormal thing I was able to find in your logs is
- I don't think this should keep C* hanging.Do you have something on kernel logs ?Do you have monitor any metrics like disk throughput / heap used / cpu load / iowait which are known as being bottlenecks / pertinent metrics ?Alain2013/8/13 Naresh Yadav <email@example.com>
PLEASE HELP ME OUT, I AM NOT ABLE TO PROCEED FROM 1 week...Load on cassandra :http://pastebin.com/DeShpHtPProblem :I have single node cassandra using CQL using datastax java driver 1.0.1 and cassandra verison 1.2.6.Hi All,
Infrastructure : 16GB machine with 8GB heap given to cassandra, i7 processor.. DEFAULT cassandra.yaml no change done by me.
-Xmx12G^ no other change in cassandra.bat
cassandra Freezes after some writes and i see no action on cassandra console for an hour...all Native_Transport threads are also killed....my program keeps running NO ERROR comes.......when i connect with cql that works........ In start it creates 16 NativeTransport threads and after 10-15 minutes Total threads goes to 128...Just before it hangs, With JCONSOLE when i see Native_Transport threads then i find most of them in state as :
i am running a usecase which stores Combinations(my project terminology) in cassandra....Currently testing storing 2.5 lakh combinations with 100 parallel threads..each thread storing one combination...real case i need to support of many CRORES but that would need different hardware and multi node cluster...
In Storing ONE combination takes around 2sec and involves :
527 INSERT INTO queries ....
506 UPDATE queries ....
954 SELECT queries ....
100 parallel threads parallely storing 100 combinations....MY CASSANDRA LOGS :
Please look last 100-200 lines of log because that is time it freezed