lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lu <chris...@gmail.com>
Subject Re: Lucene search is very slow
Date Fri, 21 Oct 2005 08:16:05 GMT
You can create the new instance of IndexSearcher right after you
finish updating the index.
And when the new IndexSearcher is ready, switch to the new one and
close the old one.

I use Hits object to do pagination too. No problem there.

Chris
----------------------------------
Full-Text Search on Any Databases
http://www.dbsight.net

On 10/20/05, Harini Raghavan <harini.raghavan@insideview.com> wrote:
> I am using Quartz jobs to update the index with the new data and this
> runs on a daily basis. So I need to figure out when exactly I should
> create a new instance of IndexSearcher.
> I had one more concern. I am not calling closing the searcher anywhere,
> would this be causing any performance bottlenecks? Also I am returning
> the Hits object to the servlet so as to do pagination. I read somewhere
> that caching the Hits object could also cause performance issue. Any ideas?
>
> -Harini
>
> Chris Lu wrote:
>
> >Yes. The best approach is not to create index searcher for every search.
> >
> >To get the updated documents, you may use an back ground thread to
> >check for new documents, or simple create an new index searcher
> >periodically. And then switch the new index searcher with the old one.
> >
> >This sounds complicated. But to my experience when writing DBSight,
> >this is the best approach.
> >
> >There is a lot of caching data in each index searcher. For large
> >index, it's definitely a waste to re-create index searcher every time.
> >
> >Chris
> >----------------------------------
> >Full-Text Search on Any Databases
> >http://www.dbsight.net
> >
> >On 10/20/05, Harini Raghavan <harini.raghavan@insideview.com> wrote:
> >
> >
> >>Hi Chris,
> >>I am initializing IndexSearcher every time the  search is executed.  The
> >>below is the search related code that is in a singleton class:
> >>
> >>public Hits searchDocuments(DocumentSearchCriteria searchCriteria, List
> >>companies) throws ApplicationException {
> >>        Hits hits = null;
> >>        String indexLoc = luceneConfig.getIndexDir();
> >>        Directory fsDir = getIndexDirectory(indexLoc, false);
> >>        IndexSearcher searcher = null;
> >>        try {
> >>             searcher = new IndexSearcher(dir);
> >>        } catch(IOException ex) {
> >>            logger.error("Exception occurred while getting the
> >>IndexSearcher.",ex);
> >>            throw new ApplicationException(ex);
> >>        }
> >>
> >>        Query query = indexSearchUtil.getSearchQuery(searchCriteria,
> >>companies);
> >>
> >>        SortField dateField = new
> >>SortField(IndexSearchConstants.DATE_FOR_SORTING, true);
> >>        SortField companyNameField = new
> >>SortField(IndexSearchConstants.COMPANY_NAME_FOR_SORTING, false);
> >>        SortField titleField = new
> >>SortField(IndexSearchConstants.TITLE_FOR_SORTING, false);
> >>
> >>        SortField[] sortFields = {dateField, companyNameField, titleField};
> >>        try {
> >>            hits = searcher.search(query, new Sort(sortFields));
> >>            logger.info("Found " + hits.length() + " document(s) that
> >>matched query '" + query + "':");
> >>        } catch(Exception e) {
> >>            logger.error("Exception occurred in searchDocuments()", e);
> >>            throw new ApplicationException(e);
> >>        }
> >>        return hits;
> >>    }
> >>
> >>I do not close the IndexSearcher anywhere. Do you think I should not
> >>create a new instance of IndexSearcher every time? Also, the index gets
> >>updated every day with the latest document. So will the IndexSearcher
> >>get the latest documents if I do not initialize it every time the search
> >>is executed?
> >>
> >>-Harini
> >>
> >>Chris Lu wrote:
> >>
> >>
> >>
> >>>Harini,
> >>>
> >>>Did you close the IndexReader every time your search is finished?
> >>>If so, 10G data will take a long time to warm up the IndexReader.
> >>>
> >>>Chris
> >>>----------------------------------
> >>>Full-Text Search on Any Databases
> >>>http://www.dbsight.net
> >>>
> >>>On 10/10/05, Koji Sekiguchi <koji.sekiguchi@m4.dion.ne.jp> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>Is it really the part of Lucene slow?
> >>>>Please take thread dumps every 15 secs, 3 to 4 times.
> >>>>What can you look at them?
> >>>>
> >>>>Koji
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>-----Original Message-----
> >>>>>From: Harini Raghavan [mailto:harini.raghavan@insideview.com]
> >>>>>Sent: Tuesday, October 11, 2005 12:38 AM
> >>>>>To: java-user@lucene.apache.org
> >>>>>Subject: Lucene search is very slow
> >>>>>
> >>>>>
> >>>>>Hi,
> >>>>>I am using lucene for search functionality in my j2ee application
using
> >>>>>JBoss as app server. The lucene index directory size is almsot 10G.
The
> >>>>>performance has been quite good until now. But after the last deploy,
> >>>>>when the server was restarted , the lucene search has become very
slow.
> >>>>>It takes almost 3-4 mins for simple searches on the index. There
has
> >>>>>been no changes to the application in the new build.
> >>>>>Does anyone have an idea what could be wrong?
> >>>>>Thanks,
> >>>>>Harini
> >>>>>
> >>>>>---------------------------------------------------------------------
> >>>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>>>For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>---------------------------------------------------------------------
> >>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>>For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>---------------------------------------------------------------------
> >>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message