lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lu <chris...@gmail.com>
Subject Re: Lucene search is very slow
Date Thu, 20 Oct 2005 15:16:44 GMT
Yes. The best approach is not to create index searcher for every search.

To get the updated documents, you may use an back ground thread to
check for new documents, or simple create an new index searcher
periodically. And then switch the new index searcher with the old one.

This sounds complicated. But to my experience when writing DBSight,
this is the best approach.

There is a lot of caching data in each index searcher. For large
index, it's definitely a waste to re-create index searcher every time.

Chris
----------------------------------
Full-Text Search on Any Databases
http://www.dbsight.net

On 10/20/05, Harini Raghavan <harini.raghavan@insideview.com> wrote:
> Hi Chris,
> I am initializing IndexSearcher every time the  search is executed.  The
> below is the search related code that is in a singleton class:
>
> public Hits searchDocuments(DocumentSearchCriteria searchCriteria, List
> companies) throws ApplicationException {
>         Hits hits = null;
>         String indexLoc = luceneConfig.getIndexDir();
>         Directory fsDir = getIndexDirectory(indexLoc, false);
>         IndexSearcher searcher = null;
>         try {
>              searcher = new IndexSearcher(dir);
>         } catch(IOException ex) {
>             logger.error("Exception occurred while getting the
> IndexSearcher.",ex);
>             throw new ApplicationException(ex);
>         }
>
>         Query query = indexSearchUtil.getSearchQuery(searchCriteria,
> companies);
>
>         SortField dateField = new
> SortField(IndexSearchConstants.DATE_FOR_SORTING, true);
>         SortField companyNameField = new
> SortField(IndexSearchConstants.COMPANY_NAME_FOR_SORTING, false);
>         SortField titleField = new
> SortField(IndexSearchConstants.TITLE_FOR_SORTING, false);
>
>         SortField[] sortFields = {dateField, companyNameField, titleField};
>         try {
>             hits = searcher.search(query, new Sort(sortFields));
>             logger.info("Found " + hits.length() + " document(s) that
> matched query '" + query + "':");
>         } catch(Exception e) {
>             logger.error("Exception occurred in searchDocuments()", e);
>             throw new ApplicationException(e);
>         }
>         return hits;
>     }
>
> I do not close the IndexSearcher anywhere. Do you think I should not
> create a new instance of IndexSearcher every time? Also, the index gets
> updated every day with the latest document. So will the IndexSearcher
> get the latest documents if I do not initialize it every time the search
> is executed?
>
> -Harini
>
> Chris Lu wrote:
>
> >Harini,
> >
> >Did you close the IndexReader every time your search is finished?
> >If so, 10G data will take a long time to warm up the IndexReader.
> >
> >Chris
> >----------------------------------
> >Full-Text Search on Any Databases
> >http://www.dbsight.net
> >
> >On 10/10/05, Koji Sekiguchi <koji.sekiguchi@m4.dion.ne.jp> wrote:
> >
> >
> >>Is it really the part of Lucene slow?
> >>Please take thread dumps every 15 secs, 3 to 4 times.
> >>What can you look at them?
> >>
> >>Koji
> >>
> >>
> >>
> >>>-----Original Message-----
> >>>From: Harini Raghavan [mailto:harini.raghavan@insideview.com]
> >>>Sent: Tuesday, October 11, 2005 12:38 AM
> >>>To: java-user@lucene.apache.org
> >>>Subject: Lucene search is very slow
> >>>
> >>>
> >>>Hi,
> >>>I am using lucene for search functionality in my j2ee application using
> >>>JBoss as app server. The lucene index directory size is almsot 10G. The
> >>>performance has been quite good until now. But after the last deploy,
> >>>when the server was restarted , the lucene search has become very slow.
> >>>It takes almost 3-4 mins for simple searches on the index. There has
> >>>been no changes to the application in the new build.
> >>>Does anyone have an idea what could be wrong?
> >>>Thanks,
> >>>Harini
> >>>
> >>>---------------------------------------------------------------------
> >>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>
> >>>
> >>>
> >>>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >>
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message