cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Leaked FDs
Date Fri, 13 Nov 2009 02:23:13 GMT
This is fixed in trunk and the 0.4 branch but did not make 0.4.2.

On Thu, Nov 12, 2009 at 6:28 PM, elsif <elsif.then@gmail.com> wrote:
> Hello.  I am testing out Cassandra and ran out of file descriptors while
> attempting to scan through the key space.  I have ~180 million unique
> keys, 10 nodes, and a replication factor of 3.
>
> Cassandra reported the following:
>
> WARN [GMFD:1] 2009-11-12 16:07:24,961 MessagingService.java (line 393)
> Exception was generated at : 11/12/2009 16:07:24 on thread GMFD:1
> Too many open files
> java.net.SocketException: Too many open files
>        at sun.nio.ch.Net.socket0(Native Method)
>        at sun.nio.ch.Net.socket(Unknown Source)
>        at sun.nio.ch.DatagramChannelImpl.<init>(Unknown Source)
>        at sun.nio.ch.SelectorProviderImpl.openDatagramChannel(Unknown
> Source)
>        at java.nio.channels.DatagramChannel.open(Unknown Source)
>        at
> org.apache.cassandra.net.UdpConnection.init(UdpConnection.java:49)
>        at
> org.apache.cassandra.net.MessagingService.sendUdpOneWay(MessagingService.java:388)
>        at
> org.apache.cassandra.gms.GossipDigestSynVerbHandler.doVerb(Gossiper.java:889)
>        at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:38)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>        at java.lang.Thread.run(Unknown Source)
>
> Lsof reports that the java process has 65486 files open (I have ulimit
> -n 65535 set in cassandra.in.sh).  Many of the lsof entries include a
> trailing '(deleted)' comment after the file path.
>
> This appears to be similar to CASSANDRA-283.  Anyone have a work around
> for this?  Would a forced GC take care of the ones marked deleted?
>
> Here is my sample code to count the number of keys:
>
> public class CClient
> {
>    public static void main(String[] args)
>    throws TException, InvalidRequestException, UnavailableException,
> UnsupportedEncodingException, NotFoundException
>    {
>        TTransport tr = new TSocket("localhost", 9160);
>        TProtocol proto = new TBinaryProtocol(tr);
>        Cassandra.Client client = new Cassandra.Client(proto);
>        tr.open();
>
>        int     count = 0;
>        int     block = 1000;
>        String  key   = " ";
>
>        while (true)
>        {
>            List<String> list = client.get_key_range("Keyspace1",
>                "Standard1", key, "~", block, ConsistencyLevel.ONE);
>            int size = list.size();
>            if (size == 0)
>                break;
>            count += size;
>            key = list.get(size - 1) + '~';
>            System.out.println("Count: " + Integer.toString(count));
>       }
>        tr.close();
>    }
> }
>
>

Mime
View raw message