Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Message-ID: <16804075.83621276887264454.JavaMail.jira@thor>
Date: Fri, 18 Jun 2010 14:54:24 -0400 (EDT)
From: "Yonik Seeley (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Subject: [jira] Commented: (SOLR-1911) File descriptor leak while indexing,
 may cause index corruption
In-Reply-To: <14098715.20791273769923186.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SOLR-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880283#action_12880283 ] 

Yonik Seeley commented on SOLR-1911:
------------------------------------

bq. Not able to repeat the problem with a build from branch_3x done earlier today

Interesting... but you see the problem with trunk still?

> File descriptor leak while indexing, may cause index corruption
> ---------------------------------------------------------------
>
>                 Key: SOLR-1911
>                 URL: https://issues.apache.org/jira/browse/SOLR-1911
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.5
>         Environment:         Ubuntu Linux, Java build 1.6.0_16-b01
>         Solr Specification Version: 3.0.0.2010.05.12.16.17.46
> 	Solr Implementation Version: 4.0-dev exported - simon - 2010-05-12 16:17:46  -- bult from updated trunk
> 	Lucene Specification Version: 4.0-dev
> 	Lucene Implementation Version: 4.0-dev exported - 2010-05-12 16:18:26
> 	Current Time: Thu May 13 12:21:12 EDT 2010
> 	Server Start Time:Thu May 13 11:45:41 EDT 2010
>            Reporter: Simon Rosenthal
>            Priority: Critical
>         Attachments: indexlsof.tar.gz, openafteropt.txt
>
>
> While adding documents to an already existing index using this build, the number of open file descriptors increases dramatically until the open file per-process limit is reached (1024) , at which point there are error messages in the log to that effect. If the server is restarted the index may be corrupt
> commits are handled by autocommit every 60 seconds or 500 documents (usually the time limit is reached first). 
> mergeFactor is 10.
> It looks as though each time a commit takes place, the number of open files  (obtained from " lsof -p `cat solr.pid` | egrep ' [0-9]+r ' ") increases by 40, There are several open file descriptors associated with each file in the index.
> Rerunning the same index updates with an older Solr (built from trunk in Feb 2010) doesn't show this problem - the number of open files fluctuates up and down as segments are created and merged, but stays basically constant.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org