cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benedict Elliott Smith <belliottsm...@datastax.com>
Subject Re: TTransportException (java.net.SocketException: Broken pipe)
Date Thu, 17 Jul 2014 09:34:04 GMT
Well with 4k maximum open files that still looks to be your culprit :)

I suggest you increase the size of your CL segments; the default is 32Mb,
and this is probably too small for the size of record you are writing. I
suspect that a 'too many open files' exception is crashing a flush which
then causes the application to get into an inconsistent state wrt commit
log reclaim.


On Thu, Jul 17, 2014 at 10:42 AM, Bhaskar Singhal <bhaskarsinghal@yahoo.com>
wrote:

> Yes, I am.
> lsof lists around 9000 open file handles.. and there were around 3000
> commitlog segments.
>
>
>   On Thursday, 17 July 2014 1:24 PM, Benedict Elliott Smith <
> belliottsmith@datastax.com> wrote:
>
>
> Are you still seeing the same exceptions about too many open files?
>
>
>
>
> On Thu, Jul 17, 2014 at 6:28 AM, Bhaskar Singhal <bhaskarsinghal@yahoo.com
> > wrote:
>
> Even after changing ulimits and moving to the recommended production
> settings, we are still seeing the same issue.
>
> root@lnx148-76:~# cat /proc/17663/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max cpu time              unlimited            unlimited            seconds
> Max file size             unlimited            unlimited            bytes
> Max data size             unlimited            unlimited            bytes
> Max stack size            8388608              unlimited            bytes
> Max core file size        0                    unlimited            bytes
> Max resident set          unlimited            unlimited            bytes
> Max processes             256502               256502
> processes
> Max open files            4096                 4096                 files
> Max locked memory         65536                65536                bytes
> Max address space         unlimited            unlimited            bytes
> Max file locks            unlimited            unlimited            locks
> Max pending signals       256502               256502               signals
> Max msgqueue size         819200               819200               bytes
> Max nice priority         0                    0
> Max realtime priority     0                    0
> Max realtime timeout      unlimited            unlimited            us
>
>
> Regards,
> Bhaskar
>
>
>   On Thursday, 10 July 2014 12:09 AM, Robert Coli <rcoli@eventbrite.com>
> wrote:
>
>
>  On Tue, Jul 8, 2014 at 10:17 AM, Bhaskar Singhal <
> bhaskarsinghal@yahoo.com> wrote:
>
> But I am wondering why does Cassandra need to keep 3000+ commit log
> segment files open?
>
>
> Because you are writing faster than you can flush to disk.
>
> =Rob
>
>
>
>
>
>
>

Mime
View raw message