cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Herron (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4571) Strange permament socket descriptors increasing leads to "Too many open files"
Date Tue, 16 Oct 2012 04:13:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476710#comment-13476710
] 

Chris Herron commented on CASSANDRA-4571:
-----------------------------------------

FYI was able to reproduce the symptom on Cassandra 1.1.6.
@[~jbellis] Re: CASSANDRA-4740 and whether it relates to this: 
* Haven't looked across all nodes for phantom connections yet
* Have searched across all logs - found a single instance of "Timed out replaying hints".
* Mina mentioned that "Nodes running earlier kernels (2.6.39, 3.0, 3.1) haven't exhibited
this". We are seeing this on Linux kernel 2.6.35 with Java 1.6.0_35.

                
> Strange permament socket descriptors increasing leads to "Too many open files"
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4571
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4571
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.1
>         Environment: CentOS 5.8 Linux 2.6.18-308.13.1.el5 #1 SMP Tue Aug 21 17:10:18
EDT 2012 x86_64 x86_64 x86_64 GNU/Linux. 
> java version "1.6.0_33"
> Java(TM) SE Runtime Environment (build 1.6.0_33-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03, mixed mode)
>            Reporter: Serg Shnerson
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 1.1.5
>
>         Attachments: 4571.txt
>
>
> On the two-node cluster there was found strange socket descriptors increasing. lsof -n
| grep java shows many rows like"
> java       8380 cassandra  113r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  114r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  115r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  116r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  117r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  118r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  119r     unix 0xffff8101a374a080            938348482 socket
> java       8380 cassandra  120r     unix 0xffff8101a374a080            938348482 socket
> " And number of this rows constantly increasing. After about 24 hours this situation
leads to error.
> We use PHPCassa client. Load is not so high (aroud ~50kb/s on write). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message