lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Underwood <timunderw...@gmail.com>
Subject Re: Healthcheck. Too many open files
Date Tue, 13 Apr 2010 00:40:37 GMT
I'm using HAProxy with 5 second healthcheck intervals and haven't seen
any problems on Solr 1.4.

My HAProxy config looks like this:

listen solr :5083
  option httpchk GET /solr/parts/admin/ping HTTP/1.1\r\nHost:\ www
  server solr01 192.168.0.101:9983 check inter 5000
  server solr02 192.168.0.102:9983 check inter 5000

Have you tried hitting /admin/ping (which handles checking for the
existence of your health file) instead of
/admin/file?file=healthcheck.txt?

-Tim

On Sat, Apr 10, 2010 at 9:26 PM, Blargy <zmanods@hotmail.com> wrote:
>
> Lance,
>
> We have have thousands of searches per minute so a minute of downtime is out
> of the question. If for whatever reason one of our solr slaves goes down I
> want to remove it ASAP from the loadbalancers rotation, hence the 2 second
> check.
>
> Maybe I am doing something wrong but the my HAProxy healthcheck is as
> follows:
> ...
> option  httpchk GET /solr/items/admin/file?file=healthcheck.txt
> ...
> so basically I am requesting that file to determine if that particular slave
> is up or not. Is this the preferred way of doing this? I kind of like the
> "Enable/Disable" feature of this healthcheck file.
>
> You mentioned:
>
> "It should not run out of file descriptors from doing this. The code
> does a 'new File(healthcheck file name).exists()' and throws away the
> descriptor. This should not be a resource leak for file desciptors."
>
> yet if i run the following on the command line:
> # lsof -p xxxx
> Where xxx is the pid of the solr, I get the following output:
>
> ...
> java    4408 root  220r   REG               8,17  56085252  817639
> /var/solr/home/items/data/index/_4y.tvx
> java    4408 root  221r   REG               8,17  10499759  817645
> /var/solr/home/items/data/index/_4y.tvd
> java    4408 root  222r   REG               8,17 296791079  817647
> /var/solr/home/items/data/index/_4y.tvf
> java    4408 root  223r   REG               8,17   7010660  817648
> /var/solr/home/items/data/index/_4y.nrm
> java    4408 root  224r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  225r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  226r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  227r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  228r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  229r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  230r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> java    4408 root  231r   REG               8,17         0  817622
> /var/solr/home/items/conf/healthcheck.txt
> ... at it keeps going ....
>
> and I've see it as high as 3000. I've had to update my ulimit to 10000 to
> overcome this problem however I feel this is really just a bandaid to a
> deeper problem.
>
> Am I doing something wrong (Solr or HAProxy) or is this a possible resource
> leak?
>
> Thanks for any input!
> --
> View this message in context: http://n3.nabble.com/Healthcheck-Too-many-open-files-tp710631p711141.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Mime
View raw message