lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: PrefixQuery rewrite() bug, ignores max clause count
Date Fri, 14 Jul 2006 23:16:00 GMT

: I dug into the code I found where the PrefixQuery rewrites itself as a
: BooleanQuery.  Unfortunately, it doesn't respect the maxClauseCount of
: the BooleanQuery in the process.  Thus, when we hit a sufficiently large
: number of results, this causes the TooManyClauses exception that a
: number of people have posted about.

this is the expected behavior of a PrefixQuery ... the
TooManyClausesException is explicitly thrown to ensure that a query
consuming too many resources isn't built, and your App is free to do
whatever you want in that situation.

: I am no expert, but I suspect all that is needed is to watch for the max
: clause count and then 'chunk' the boolean query.  I think the following
: should work (changes in blue):

1) colorzing your mail doesn't play nicely with the mailing list ...
sending "diffs" is the prefered way to show potential changes to the code.

2) assuming i understand what it is you've changed, you've "worked
arround" the TooManyClausesException in a somewhat complicated way (you
could just as easily increase the maxClauseCount and save yourslef some
headache) ... by doing this you bypass the whole point of having a
maClauseCount: to prevent a query that will consume too many resources
(namely RAM and query execution time) from being constructed.

3) your change modifies the core impacts from each of the Terms that match
the prefix ... if you don't care about the score impacts from the terms,
then there are other options.  Solr has a PrefixFilter class which can be
wrapped in a ConstantScoreQuery to support prefix queries regardless of
your term distribution.


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message