lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Question on wildcard queries, filters, scoring and TooManyClauses exception
Date Fri, 16 Aug 2013 13:03:51 GMT
I can't explain all of it and 3.0 is way old ... you might like to
think about upgrading.

However in your first snippet you don't need the query AND the filter.
 Either one will suffice.  In some circumstances, as you say, filters
are preferable but queries and filters are often interchangeable.

On the rest of it, I don't know what is going on.  What java classes
are you getting back from QueryParser?  Giving it a variable name of
prefixQuery doesn't make it so - what does
prefixQuery.getClass().getName() say?


--
Ian.


On Thu, Aug 15, 2013 at 6:43 PM, Bill Chesky
<Bill.Chesky@learninga-z.com> wrote:
> Hello,
>
> I know this is a perennial question here because I've spent a lot of time searching for
an answer.  I've seen the discussions about the TooManyClauses exception and I understand
generally why you get the it.  I see lots of discussion about using filters to avoid it but
I still can't get it to work.  I think I'm just missing something fundamental.
>
> I'm using Lucene 3.0.
>
> I'm trying to do prefix queries on an index.  I figured there might be times where I
might run into the TooManyClauses exception so from reading discussions on the issue I figured
I should use a filter.  I found the PrefixFilter class and began experimenting with it.  E.g.
this works:
>
> QueryParser queryParser = new QueryParser(Version.LUCENE_30, "my_field", new StandardAnalyzer(Version.LUCENE_30));
> Query prefixQuery = queryParser.parse("t*");
> PrefixFilter prefixFilter = new PrefixFilter(new Term("my_field", "t"));
> indexSearcher.search(prefixQuery, prefixFilter, collector);
>
> This returns about 5000 hits on my index.
>
> But then I discovered that it works just as well without the filter:
>
> QueryParser queryParser = new QueryParser(Version.LUCENE_30, "my_field", new StandardAnalyzer(Version.LUCENE_30));
> Query prefixQuery = queryParser.parse("t*");
> indexSearcher.search(prefixQuery, collector);
>
> Why, I don't know.  Seems like this would get expanded out into 5000 BooleanQueries and
since my max clause count is still set to the default 1024 I should get the exception.  But
I didn't.  So maybe I don't need the filter after all?
>
> Next, I need scoring to work.  I read that with wildcard queries all scores are set to
1.0 by default.  But I read you can use the QueryParser.setMultiTermRewriteMethod() method
to take scoring into account again.  So I tried:
>
> QueryParser queryParser = new QueryParser(Version.LUCENE_30, "my_field", new StandardAnalyzer(Version.LUCENE_30));
> queryParser.setMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_REWRITE);
> Query prefixQuery = queryParser.parse("t*");
> indexSearcher.search(prefixQuery, collector);
>
> Now, I get the TooManyClauses exception.
>
> I tried adding the PrefixFilter back in but with no luck.  Still get the exception.
>
> Again, sorry if this has been discussed before.  Just not seeing an answer to this after
much searching and I just don't understand what is going on here.  Any help appreciated. 
Links welcome.
>
> Bill
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message