lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vasu shah <vasusha...@yahoo.com>
Subject Re: PrefixFilter Memory Consumption
Date Tue, 17 Oct 2006 22:31:43 GMT
Thanks for the explanation. 

I am using ChainedFilter and it is taking some more time than using just one Filter.

I read somewhere on the lucene forums that the speed can be increased for Filters if we have
a large bitset and then work on it. Is it possible and if yes, how? I would like to know the
technique.

Thanks once again.
-Vasu

Yonik Seeley <yonik@apache.org> wrote: On 10/17/06, vasu shah  wrote:
>   Can anyone please tell as to what is the difference between PrefixFilter and WildcardQuery
as far as memory is concerned?
>
>   I saw the code of PrefixFilter and it gets TermEnum for all the terms in the index.
Won't this consume memory??

It takes time, not memory.  TermEnum is like an iterator - it doesn't
keep everything it iterates over in memory.  The memory for filters
will be for the bitset (1 bit for every document in the index, so a 1M
doc index will take 125KB of mem per filter).

Both PrefixQuery and PrefixFilter need to use a TermEnum, the
difference is what they do with the resulting terms.

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



 		
---------------------------------
 All-new Yahoo! Mail - Fire up a more powerful email and get things done faster.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message