lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charlie Smith" <Smit...@ldschurch.org>
Subject Re: too many files open error
Date Tue, 30 Mar 2004 17:05:13 GMT
Thanks for the information.  I downloaded 1.3-rc2 and put a IndexReader.close()
at the end of the search
routine.  This seems to have cleared up the problems.   Also, demo source code
for results.jsp to 
return a pointer to IndexReader so that it could be closed at end of search.
Ie.
          searcher = new IndexSearcher(ir =
                        IndexReader.open(indexName));     //create an
indexSearcher for our page
...
ir.close();


>>>> erik 3/27/2004 4:44:28 AM >>>
>On Mar 27, 2004, at 1:28 AM, Charlie Smith wrote:
>> What would be the URL for the JUnit stuff?
>
>Look in the src/test directory of where you checked out Lucene.  All
>JUnit tests live there and below.
>
>> BTW: I was able to build a new Index.class file, with the additional
>> line
>> iw.setUserCompoundFile(true)  after extracting the
>> lucene-1.4-rc1-dev.jar.
>>
>> Then reindexed.  Guess what - no worky.  :(
>
>Maybe you'd care to share some *technical* details to elaborate on "no
>worky"?!

Still get the "too many files open" error on invoking a modified results.jsp. 
(The one that comes with Lucene.)
The index is created with a call to the IndexWriter.class file.   The
Index.class file calls IndexWriter,
and I modified to have the setUseCompoundFile(true).  Added lines 350 and 442
as suggested.

>
>What Index.class are you talking about?  The demo application?
>
>> Can I get 1.3-RC2?  Could someone point me to the URL for this
>> download please
>> ;)
>
>Use CVS :)
>
>> I noticed following entry in mail archives:
>> http://www.mail-archive.com/lucene-user@jakarta.apache.org/ 
>> msg06118.html
>>
>> along with 139 others that dealt with the "too many files open"
>> problem.
>>
>> Looks like this is a high priority problem that might justify a new
>> release in
>> and of itself?
>
>People have been using Lucene for years and managing the file handle
>issue by setting ulimit and other tricks like optimizing to reduce the
>number of segments.  So it is not as much a problem as it is a known
>issue that can be managed.
>

My ulimit is set to unlimited.
>From what I can tell, it is a stress test issue that seems to work under
1.3-rc2.  Would anyone understand the differences to know if it will work as
well under next stable release of  Lucene?


>I'm not up to speed on what the issues with 1.3 final are - I've just
>started hearing about it.  Is there a reproducible example that
>demonstrates a problem?
>
>       Erik
>

John Brown has made his source available.  Go to Google and search for
docSearcher.  He seems quite willing to help where needed.  Use the reults.jsp
routine that comes with Lucene to test, with following changes:
<snip>
<                 Analyzer analyzer = new StopAnalyzer();              
//construct our usual analyzer
---
>                 Analyzer analyzer = new StandardAnalyzer();              
//construct our usual analyzer
68,69c54,56
<                         query = QueryParser.parse(queryString, "contents",
analyzer); //parse the
<                 } catch (ParseException e) {                          //query
and construct the Query
---
>                         query = QueryParser.parse(queryString, "body",
analyzer); //parse the
>                         //query = query.rewrite(reader);
>               } catch (ParseException e) {                          //query
and construct the Query
87a75


>                 <tr><td><font size=5>Search results for </font><font
size=5
color="blue"><%=queryString%></td></tr>
>                 <tr><tr>
>
108a96,97
>                 // cws: 2/25/04 added this to get format href link.
>                 RE r = new RE("/path/to/site/root/");
111d99
<                 <tr>
114,122c102,131
<                         String doctitle = doc.get("title");            //get
its title
<                         String url = doc.get("url");                   //get
its url field
<                         if ((doctitle == null) || doctitle.equals("")) //use
the url if it has no title
<                                 doctitle = url;
<                                                                        //then
output!
< %>
<                         <td><a href="<%=url%>"><%=doctitle%></a></td>
<                         <td><%=doc.get("summary")%></td>
<                 </tr>
---
>                         String path = doc.get("path");
>                         String type = doc.get("type");
>                         String title = doc.get("title");
>
>                         // cws: 2/25/04 added this to get format href link.
>                         String path_part =  r.subst(path, "/");
>
>                         String summary = doc.get("summary");
>                         String size = doc.get("size");
>                         String date = doc.get("mod_date");
>
>                         // date formating
>                       java.util.Date bd=DateField.stringToDate(date);
>                       Calendar nowD=Calendar.getInstance();
>                       nowD.setTime(bd);
>                       int mon=nowD.get(nowD.MONTH)+1;
>                       int year=nowD.get(nowD.YEAR);
>                       int day=nowD.get(nowD.DAY_OF_MONTH);
>                       date = mon+"/"+day+"/"+year;
>
>                         // size formating
>                       float floatSize= (Float.parseFloat(size)/1000);
>
>                       // print output
>                       %>
>                         <tr><td><a
href="<%=path_part%>"><%=title%></a></td></tr>
>                         <tr><td><%=summary%></td></tr>
>                         <tr><td><I><font color="green" size="2"><%=date%>
<%=floatSize%> K bytes</font></I></td></tr>
>                         <tr><td><font color="gray" size="2"><%=path%>
&nbsp;<%=path_part%> &nbsp;<%=title%></font></td></tr>

</snip>


>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org 
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org 
>
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message