lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: Read large size index
Date Mon, 29 Jun 2009 13:53:32 GMT
Hey there, that makes things easier. :)

ok here are some questions:

Do you iterate over all docs calling hits.doc(i) ?If so do you have to
load all fields to render your results, if not you should not retrieve
all of them?
You use IndexSearchersearch(Query q,...) which returns a Hits object
have you tried to use the new search methods returning TopDocs?

when you search for pdf and get 30k results you load all the "stored"
field content into memory once you call IndexSearcher.doc(i) as it
internally calls IndexReader.document(i, null). This is equivalent to
a "Load All fields" FieldSelector.
You can have a closer look at FieldSelector and the new search methods
which accept them. This is a way to make you retrieval faster and load
only the fields you really need.



On Mon, Jun 29, 2009 at 3:31 PM, m.harig<m.harig@gmail.com> wrote:
>
> Thanks again,
>
>
>       Did i index my files correctly, please need some tips, the following
> is the error when i run my keyword , i typed pdf , thats it , because i've
> got around 30,000 files named pdf,
>
>
> HTTP Status 500 -
>
>
>
> type Exception report
>
>
>
> message
>
>
>
> description The server encountered an internal error () that prevented it
> from fulfilling this request.
>
>
>
> exception
>
>
>
> javax.servlet.ServletException: Servlet execution threw an exception
>
>
>
> root cause
>
>
>
> java.lang.OutOfMemoryError: Java heap space
>
>        java.util.Arrays.copyOfRange(Unknown Source)
>
>        java.lang.String.<init>(Unknown Source)
>
>        org.apache.lucene.store.IndexInput.readString(IndexInput.java:113)
>
>        org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:324)
>
>        org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:166)
>
>        org.apache.lucene.index.SegmentReader.document(SegmentReader.java:659)
>
>        org.apache.lucene.index.IndexReader.document(IndexReader.java:525)
>
>        org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:92)
>
>        org.apache.lucene.search.Hits.doc(Hits.java:167)
>
>        com.npedia.liteSearch.helper.SearchHelper.getResults(SearchHelper.java:103)
>
>
> com.npedia.liteSearch.servlet.SearchServlet.doProcess(SearchServlet.java:164)
>
>        com.npedia.liteSearch.servlet.SearchServlet.doGet(SearchServlet.java:39)
>
>        javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>
>        javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
>
>
>
> note The full stack trace of the root cause is available in the Apache
> Tomcat/6.0.10 logs.
>
> Apache Tomcat/6.0.10
> --
> View this message in context: http://www.nabble.com/Read-large-size-index-tp24251993p24254191.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message