lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin.St...@materna.de
Subject AW: Understanding TooManyClauses-Exception and Query-RAM-size
Date Mon, 12 Jul 2004 09:42:42 GMT
Hi Kevin,

thanks for your answer. That could really solve the problem with the
modificationDate or similar fields.

But what if you create queries that ultimately return only a few hits but
contain a RangeQuery that searches for example an ID-Field of some kind,
where you have to cover a wide range of IDs? I think in general, you will
always have fields that contain lots of different terms and searching even a
small range of one of these fields may lead to this Exception. 

The bottom line in my opinion is, that you have to take care for yourself,
not to create certain type of queries that could lead to this Exception. The
type of query completely depends on the index which means as the index grows
you have to restrict the ranges of more and more rangequeries.

One way would be, to catch this Exception and gracefully present a message
to the user to further restrict his query. But this could lead to some
confusion, if the user knows that he has entered some very restrictive query
in addition to some RangeQuery that internally leads to this Exception. 

What I  really would like to see are some best practices or some advice from
some users who are working with really large indices how they handle this
situation, or why they  don't have to  care about it or maybe why I am
completely missing the point ;-))


Thanks,

Martin


-----Urspr√ľngliche Nachricht-----
Von: Kevin A. Burton [mailto:burton@newsmonster.org]
Gesendet: Donnerstag, 8. Juli 2004 21:11
An: Lucene Users List
Betreff: Re: Understanding TooManyClauses-Exception and Query-RAM-size


Martin.Stein@materna.de wrote:

>Hi,
>
>a couple of weeks ago we migrated from Lucene 1.2 to 1.4rc3. Everything
went
>smoothly, but we are experiencing some problems with that new constant
limit
>
>
>	maxClauseCount=1024
>
>which leeds to Exceptions of type 
>
>	org.apache.lucene.search.BooleanQuery$TooManyClauses 
>
>when certain RangeQueries are executed (in fact, we get this Excpetion when
>we execute certain Wildcard queries, too). Although we are working with a
>fairly small index with about 35.000 documents, we encounter this Exception
>when we search for the property "modificationDate". For example
>
>	modificationDate:[000000 TO 0dwc970kw] 
>
>  
>
We talked about this the other day.

http://wiki.apache.org/jakarta-lucene/IndexingDateFields

Find out what type of precision you need and use that.  If you only need 
days or hours or minutes then use that.   Millis is just too small. 

We're only using days and have queries for just the last 7 days as max 
so this really works out well...

Kevin

-- 

Please reply using PGP.

    http://peerfear.org/pubkey.asc    
    
    NewsMonster - http://www.newsmonster.org/
    
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
  IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message