lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Vasiliev <>
Subject Re[4]: OutOfMemory with search(Query, Sort)
Date Wed, 05 Apr 2006 04:59:15 GMT
Hello Hoss,

Thanks for your answer, you're right, filepathes are pretty much
unique. Anyway I don't want this total-field-cache-loading situation occur
in any circumstances - it's too expensive. My app usually crawls while
user searches are performed. Crawl involves additions and deletions so
IndexSearcher get closed relatively frequently. Seems like Lucene
would reload the whole field cache for each new IndexSearcher, which
would be a big hit anyway. So I'll try FieldCache overriding solution
proposed by you and Yonik and may be commit it to Lucene as a patch.

Btw do I understand right that concrete FieldCache class isn't pluggable
at Lucene at the moment?

: >> sort by filePath field which can be 100 bytes at average meaning 400M
: >> RAM for the cache
CH> :
CH> : Well, it's probably not quite that bad...

CH> yeah, but in his case he's dealing with filepaths -- i'm guessing that
CH> each document represents a file, and no two files will have the same path.

CH> some benefit may be gained in spliting the filepath field up into a
CH> dirpath field and a filename field, and then sortinging on "dirpath,
CH> filename" .. this should reduce the size quite a bit if the number of

Best regards,
 Artem sharehound, the open source filesystems indexer

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message