lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: lucene java OOM while sorting more than one field
Date Wed, 27 Feb 2008 14:54:06 GMT
The first question is always "how much memory are you giving
your JVM?".

A 256M index is pretty small, I wouldn't be surprised if your JVM is using
some vary small default....

Best
Erick

On Wed, Feb 27, 2008 at 6:23 AM, GURUPRASAD MS <guruprasad.ms@gmail.com>
wrote:

>  Lucene Index contains 2.1 Million records (indexed from 2.1 million
> records
> from sqlserver DB).
> Lucene Index file Size 256MB
> Lucene version: 2.3
> Searching works fine when we sort the results on a single field. However,
> if
> the search results is sorted on more than one field we get Out of Memory
> exception.
> We restrict the number of search results to 250
>
> Out of Memory exception is quite consistent in 2.3. We recently moved from
> 2.0 to 2.3.
> Version 2.0 also used to give the OOM but not this frequent.
>
>
> *Code Snippet:*
>
> final QueryParser parser;
> StandardAnalyzer analyzer = new StandardAnalyzer();
> String sSearchQuery = "MUSIC";
> Sort oSort = null;
> public static final String G2KEYFIELDS = "G2KEYFIELDS";
> oSort = new Sort(new SortField[]{ new SortField (
> GroupsConstants.LAST_MODIFIED,SortField.INT,true), new SortField(
> GroupsConstants.GROUPNAME) });
>
> parser = new QueryParser(G2KEYFIELDS, analyzer);
> final Vector ids = new Vector();
> FSDirectory dir = null;
> IndexSearcher searcher = null;
> try {
> dir = FSDirectory.getDirectory(index);
> searcher = new IndexSearcher(dir);
> Query query = parser.parse(sSearchQuery);
> Hits hits = searcher.search(query, oSort);
> for (int i = 0; i != hits.length() && i != 250; ++i) {
> final Document doc = hits.doc;
> Integer oiGroupId=new Integer(doc.getField (GroupsConstants.IDENTITY
> ).stringValue());
> if(!ids.contains(oiGroupId)){ ids.addElement(oiGroupId); }
> }
> searcher.close();
> dir.close();
>
>
> Thanks in Advance
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message