cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "大龙哥" <522062...@qq.com>
Subject Amounts of ‘CLOSE_WAIT’ status connection occur at cassandra servers
Date Wed, 02 Sep 2015 02:38:46 GMT
Hi,

I encountered an issue recently when I built spark-cassandra applications envirenment.


Problem description:
  1. I run spark applications to process data then write result to cassandra.
  2. In the beginning, all going well. But several hours(or serval days)  later amounts of
‘CLOSE_WAIT’ status connection occur at cassandra servers.
      It leads spark-application connect cassandra  faild, also execute command 'nodetool
status' show 'DN' status on the trouble node.
  3. CLOSE_WAIT connections include communication with other cassandra nodes(use port 7000,
a small part) and communication with spark-applications(use port 9042, a large part)
  4. Restart the trouble cassandra node, recovery. 


Envirenment:
   Versions: spark 1.4.1 (spark-core 2.10)       cassandra: 2.1.8           spark-cassandra-connector:1.4.0-M3
 
   
   8 spark-applications run on 6 spark servers, and has 3 cassandra nodes.


   Dose anybody have the samilar problem? or knows the root cause.
   Thanks.




   Regards.




   dandykang
Mime
View raw message