lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fran├žois Eric" <francois.e...@jarca.ca>
Subject Query joining 2 indexes
Date Fri, 18 Dec 2009 14:08:54 GMT
Hello,

I have a performance problem and would need expert advice on how to go 
about fixing it:

I currently have 2 indexes: Daily and Hourly.  The Daily index contains 
about 1,000,000 documents and my Hourly index approximately: 24,000,000 
documents.  My Daily index contains many fields and some of them are IDs 
to my Hourly Index.
What I want to do is fetch data in one request (if possible).
Right now I do it in many requests:
1- Get the matching Daily documents (say it returns 500 documents)
2- For each of these documents, locate the Hourly Index Id and fetch it.

Therefore I make 501 requests to lucene.  This causes some performance 
issues I guess because of the overhead to  making a request to Lucene.

Is it possible to do this in 1 request?  I'm thinking no because I'm not 
sure what the result set would be but maybe I'm missing something.

If not I guess it would be possible to build a query with my 500 hourly 
ids and make a OR between them to make it in 2 requests....but then I 
have to find the matching documents.  Will this overflow if I have 50000 
ids in my query?

Anyway, I just want advice on how one would address this situation.

Thank you very much,

Fran├žois

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message