lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1911) File descriptor leak while indexing, may cause index corruption
Date Mon, 17 May 2010 17:46:43 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12868309#action_12868309
] 

Yonik Seeley commented on SOLR-1911:
------------------------------------

{quote}
Here's a clue -?? I looked for file descriptors associated with one .prx file that was present
in the original snapshot in each lsof output

grep -c _r8.prx lsof.*
lsof.0:1
lsof.1:2
lsof.2:3
lsof.3:4
lsof.4:5
lsof.5:6
{quote}

OK... this definitely sounds like a leak then.  And the fact that you're using the stock solrconfig.xml
should rule out spellchecker, replication, etc.


> File descriptor leak while indexing, may cause index corruption
> ---------------------------------------------------------------
>
>                 Key: SOLR-1911
>                 URL: https://issues.apache.org/jira/browse/SOLR-1911
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.5
>         Environment:         Ubuntu Linux, Java build 1.6.0_16-b01
>         Solr Specification Version: 3.0.0.2010.05.12.16.17.46
> 	Solr Implementation Version: 4.0-dev exported - simon - 2010-05-12 16:17:46  -- bult
from updated trunk
> 	Lucene Specification Version: 4.0-dev
> 	Lucene Implementation Version: 4.0-dev exported - 2010-05-12 16:18:26
> 	Current Time: Thu May 13 12:21:12 EDT 2010
> 	Server Start Time:Thu May 13 11:45:41 EDT 2010
>            Reporter: Simon Rosenthal
>            Priority: Critical
>         Attachments: indexlsof.tar.gz, openafteropt.txt
>
>
> While adding documents to an already existing index using this build, the number of open
file descriptors increases dramatically until the open file per-process limit is reached (1024)
, at which point there are error messages in the log to that effect. If the server is restarted
the index may be corrupt
> commits are handled by autocommit every 60 seconds or 500 documents (usually the time
limit is reached first). 
> mergeFactor is 10.
> It looks as though each time a commit takes place, the number of open files  (obtained
from " lsof -p `cat solr.pid` | egrep ' [0-9]+r ' ") increases by 40, There are several open
file descriptors associated with each file in the index.
> Rerunning the same index updates with an older Solr (built from trunk in Feb 2010) doesn't
show this problem - the number of open files fluctuates up and down as segments are created
and merged, but stays basically constant.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message