incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: File Descriptor leak
Date Sun, 13 Jun 2010 19:32:32 GMT
Can you open a new ticket, then?  Preferably with the thrift code
involved, I'm not sure what find_by_natural_key or find_all_by_service
is translating into.  (It looks like just one of those is responsible
for the leak.)

On Sun, Jun 13, 2010 at 12:11 PM, Matthew Conway <matt@backupify.com> wrote:
> Pretty sure as the list of file descriptors below shows (at this point the client has
exited, so doubly sure its not open sockets):
>
> # lsof -p `ps ax | grep [C]assandraDaemon | awk '{print $1}'` | awk '{print $9}' | sort
| uniq -c | sort -n | tail -n 5
>      2 /usr/local/apache-cassandra-2010-06-11_12-30-33/lib/slf4j-log4j12-1.5.8.jar
>      2 /usr/local/apache-cassandra-2010-06-11_12-30-33/lib/snakeyaml-1.6.jar
>      2 /usr/share/java/gnome-java-bridge.jar
>   1003 /mnt/cassandra/data/MyKeyspace/MySuperColumn-c-1-Data.db
>   1003 /mnt/cassandra/data/MyKeyspace/MySuperColumn-c-2-Data.db
>
> On Jun 11, 2010, at Fri Jun 11, 7:34 PM, Jonathan Ellis wrote:
>
>> it goes up by exactly 2000, which is the number of loop iterations
>> exactly?  are you sure this isn't just counting your open sockets?
>>
>> On Fri, Jun 11, 2010 at 1:53 PM, Matthew Conway <matt@backupify.com> wrote:
>>> Thanks, I just tried apache-cassandra-2010-06-11_12-30-33 (hudson 462) but my
tests ares still reporting a leak (though not as bad), I do the following (ruby tests using
cassandra_object/cassandra, but you should be able to get the idea):
>>>
>>>      should "not leak file descriptors" do
>>>        cassandra_pid = `ps ax | grep [C]assandraDaemon | awk '{print $1}'`
>>>        original_count = `lsof -p #{cassandra_pid}`.lines.to_a.size
>>>        assert original_count > 0
>>>        count = 1000
>>>        count.times do |n|
>>>          ChildMetadatum.new(:service_id => 4, :child_id => "def#{n}",
:updated => Time.now, :labels => ["label2", "label3"]).save!
>>>        end
>>>        count.times do |n|
>>>          ChildMetadatum.find_by_natural_key(:service_id => 4, :child_id
=> "def#{n}")
>>>          ChildMetadatum.find_all_by_service_id(3)
>>>        end
>>>        new_count = `lsof -p #{cassandra_pid}`.lines.to_a.size
>>>        assert new_count > 0
>>>        assert new_count < original_count * 1.1, "File descriptors leaked
from #{original_count} to #{new_count}"
>>>      end
>>>
>>> Which reports: File descriptors leaked from 112 to 2112.
>>> SHould I reopen the bug or create a new one?
>>>
>>> Matt
>>>
>>> On Jun 10, 2010, at Thu Jun 10, 6:40 PM, Jonathan Ellis wrote:
>>>
>>>> Fixed in https://issues.apache.org/jira/browse/CASSANDRA-1178
>>>>
>>>> On Thu, Jun 10, 2010 at 9:01 AM, Matt Conway <matt@backupify.com> wrote:
>>>>> Hi All,
>>>>> I'm running a small 4-node cluster with minimal load using
>>>>> the 2010-06-08_12-31-16 build from trunk, and its exhausting file
>>>>> descriptors pretty quickly (65K in less than an hour).  Here's a list
of the
>>>>> files I see it  leaking, I can do a more specific query if you'd like.
 Am I
>>>>> doing something wrong, is this a known problem, something being done
wrong
>>>>> from the client side, or something else?  Any help appreciated, thanks,
>>>>> Matt
>>>>> root@cassandra01:~# lsof -p `ps ax | grep [C]assandraDaemon | awk '{print
>>>>> $1}'` | awk '{print $9}' | sort | uniq -c | sort -n | tail -n 5
>>>>>       3 /mnt/cassandra/data/system/Schema-c-2-Data.db
>>>>>    1278 /mnt/cassandra/data/MyKeyspace/MyColumnType-c-7-Data.db
>>>>>    1405 /mnt/cassandra/data/MyKeyspace/MyColumnType-c-9-Data.db
>>>>>    1895 /mnt/cassandra/data/MyKeyspace/MyColumnType-c-5-Data.db
>>>>>   26655 /mnt/cassandra/data/MyKeyspace/MyColumnType-c-11-Data.db
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Jonathan Ellis
>>>> Project Chair, Apache Cassandra
>>>> co-founder of Riptano, the source for professional Cassandra support
>>>> http://riptano.com
>>>
>>>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Mime
View raw message