lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: about TopFieldDocs
Date Mon, 05 Jan 2009 13:30:09 GMT
Mostly, the difference is in the sorting. Your
example (1) scores by document relevance whereas
your example (2) sorts by whatever is in fieldName.

example (2), because it is sorting, will try to
cache all the distinct *terms* in your index for that field,
which is probably where  your out of memory is
coming from. The number of documents
is irrelevant here, what is relevant is the number of
distinct terms in your "fieldName" field.

I'd get a copy of Luke to look at your index and see if it's
what you expect.

And yes, increasing memory should help. How much memory
are you running with when you get the OOM error? Sometimes
the default memory allocation is very small and most of it's taken
up with the program leaving very little left over for caching.

Best
Erick

2009/1/5 장용석 <need4spd@gmail.com>

> Hi.. :)
>
> I have a simple question..
>
> I have two sample code.
>
> 1) TopDocCollector collector = new TopDocCollector(5 * hitsPerPage);
>    QueryParser parser = new QueryParser(fieldName, analyzer);
>    query = parser.parse("keyword");
>
>    searcher.search(query, collector);
>    ScoreDoc[] hits = collector.topDocs().scoreDocs;
>   Document doc = searcher.doc(hits[i].doc);
>
> 2)
> Sort sort = new Sort(fieldName,true);
> QueryParser parser = new QueryParser(fieldName, analyzer); query =
> parser.parse("keyword");
> TopFieldDocs tfd = searcher.search(query,null,50,sort);
>
> hits = tfd.scoreDocs;
> Document doc = searcher.doc(hits[i].doc);
>
> In that case, what is the difference with between ScoreDoc[] hits =
> collector.topDocs().scoreDocs
> and ScoreDoc[] hits = tfd.scoreDocs?
>
>
> and.. in case 2)
> It did throw exception java.lang.OutOfMemoryError: Java heap space.
> I did not set jvm option and my index size is about 1G.
> and after search collector.getTotalHits() is 2585.
>
> I thins 2585 is not many documents....
>
> What do i do for fix this problem? just increase jvm heap memory size? or
> Is there other way?
>
> I need some advice..:)
>
> Sorry for my bad English..
>
> thanks.
>
> Jang.
>
> --
> DEV용식
> http://devyongsik.tistory.com
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message