Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <4B2B8CF6.4080004@polymtl.ca>
Date: Fri, 18 Dec 2009 09:08:54 -0500
From: "=?ISO-8859-1?Q?Fran=E7ois_Eric?=" <francois.eric@jarca.ca>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Query joining 2 indexes
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Hello,

I have a performance problem and would need expert advice on how to go 
about fixing it:

I currently have 2 indexes: Daily and Hourly.  The Daily index contains 
about 1,000,000 documents and my Hourly index approximately: 24,000,000 
documents.  My Daily index contains many fields and some of them are IDs 
to my Hourly Index.
What I want to do is fetch data in one request (if possible).
Right now I do it in many requests:
1- Get the matching Daily documents (say it returns 500 documents)
2- For each of these documents, locate the Hourly Index Id and fetch it.

Therefore I make 501 requests to lucene.  This causes some performance 
issues I guess because of the overhead to  making a request to Lucene.

Is it possible to do this in 1 request?  I'm thinking no because I'm not 
sure what the result set would be but maybe I'm missing something.

If not I guess it would be possible to build a query with my 500 hourly 
ids and make a OR between them to make it in 2 requests....but then I 
have to find the matching documents.  Will this overflow if I have 50000 
ids in my query?

Anyway, I just want advice on how one would address this situation.

Thank you very much,

Fran�ois

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org