lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harini Raghavan <harini.ragha...@insideview.com>
Subject Re: Lucene search is very slow
Date Fri, 21 Oct 2005 05:02:24 GMT
I am using Quartz jobs to update the index with the new data and this 
runs on a daily basis. So I need to figure out when exactly I should 
create a new instance of IndexSearcher.
I had one more concern. I am not calling closing the searcher anywhere, 
would this be causing any performance bottlenecks? Also I am returning 
the Hits object to the servlet so as to do pagination. I read somewhere 
that caching the Hits object could also cause performance issue. Any ideas?

-Harini

Chris Lu wrote:

>Yes. The best approach is not to create index searcher for every search.
>
>To get the updated documents, you may use an back ground thread to
>check for new documents, or simple create an new index searcher
>periodically. And then switch the new index searcher with the old one.
>
>This sounds complicated. But to my experience when writing DBSight,
>this is the best approach.
>
>There is a lot of caching data in each index searcher. For large
>index, it's definitely a waste to re-create index searcher every time.
>
>Chris
>----------------------------------
>Full-Text Search on Any Databases
>http://www.dbsight.net
>
>On 10/20/05, Harini Raghavan <harini.raghavan@insideview.com> wrote:
>  
>
>>Hi Chris,
>>I am initializing IndexSearcher every time the  search is executed.  The
>>below is the search related code that is in a singleton class:
>>
>>public Hits searchDocuments(DocumentSearchCriteria searchCriteria, List
>>companies) throws ApplicationException {
>>        Hits hits = null;
>>        String indexLoc = luceneConfig.getIndexDir();
>>        Directory fsDir = getIndexDirectory(indexLoc, false);
>>        IndexSearcher searcher = null;
>>        try {
>>             searcher = new IndexSearcher(dir);
>>        } catch(IOException ex) {
>>            logger.error("Exception occurred while getting the
>>IndexSearcher.",ex);
>>            throw new ApplicationException(ex);
>>        }
>>
>>        Query query = indexSearchUtil.getSearchQuery(searchCriteria,
>>companies);
>>
>>        SortField dateField = new
>>SortField(IndexSearchConstants.DATE_FOR_SORTING, true);
>>        SortField companyNameField = new
>>SortField(IndexSearchConstants.COMPANY_NAME_FOR_SORTING, false);
>>        SortField titleField = new
>>SortField(IndexSearchConstants.TITLE_FOR_SORTING, false);
>>
>>        SortField[] sortFields = {dateField, companyNameField, titleField};
>>        try {
>>            hits = searcher.search(query, new Sort(sortFields));
>>            logger.info("Found " + hits.length() + " document(s) that
>>matched query '" + query + "':");
>>        } catch(Exception e) {
>>            logger.error("Exception occurred in searchDocuments()", e);
>>            throw new ApplicationException(e);
>>        }
>>        return hits;
>>    }
>>
>>I do not close the IndexSearcher anywhere. Do you think I should not
>>create a new instance of IndexSearcher every time? Also, the index gets
>>updated every day with the latest document. So will the IndexSearcher
>>get the latest documents if I do not initialize it every time the search
>>is executed?
>>
>>-Harini
>>
>>Chris Lu wrote:
>>
>>    
>>
>>>Harini,
>>>
>>>Did you close the IndexReader every time your search is finished?
>>>If so, 10G data will take a long time to warm up the IndexReader.
>>>
>>>Chris
>>>----------------------------------
>>>Full-Text Search on Any Databases
>>>http://www.dbsight.net
>>>
>>>On 10/10/05, Koji Sekiguchi <koji.sekiguchi@m4.dion.ne.jp> wrote:
>>>
>>>
>>>      
>>>
>>>>Is it really the part of Lucene slow?
>>>>Please take thread dumps every 15 secs, 3 to 4 times.
>>>>What can you look at them?
>>>>
>>>>Koji
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>-----Original Message-----
>>>>>From: Harini Raghavan [mailto:harini.raghavan@insideview.com]
>>>>>Sent: Tuesday, October 11, 2005 12:38 AM
>>>>>To: java-user@lucene.apache.org
>>>>>Subject: Lucene search is very slow
>>>>>
>>>>>
>>>>>Hi,
>>>>>I am using lucene for search functionality in my j2ee application using
>>>>>JBoss as app server. The lucene index directory size is almsot 10G. The
>>>>>performance has been quite good until now. But after the last deploy,
>>>>>when the server was restarted , the lucene search has become very slow.
>>>>>It takes almost 3-4 mins for simple searches on the index. There has
>>>>>been no changes to the application in the new build.
>>>>>Does anyone have an idea what could be wrong?
>>>>>Thanks,
>>>>>Harini
>>>>>
>>>>>---------------------------------------------------------------------
>>>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>>
>>>      
>>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message