lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jim shirreffs" <>
Subject Implementing lagre secure Lucene search system questions.
Date Thu, 03 May 2007 18:05:30 GMT
Hi, I'm a relative Lucene newbe and would appreciate some expert advice.

I would like to make fulltest searchable,  files distributed on various 
local hosts in the intranet. My startup plan is to index these files locally 
and then merge all the little indexes into a master indexes on a search 
host. Once complete I will delete all the little indexes and just keep the 
master index.

Then over time as new files are added to the various local hosts I'll index 
it (new file) locally and then (once again) send the little indexes to be 
merged into the master index on the search host. Once complete I will again 
delete the no longer needed little index.

I have been reading that merging indexes can be slow, my master index will 
be huge since the entire documents collection may be in the tens of 

Is this true? If so would it be better to send the (new) document to the 
search host and index it there rather then sending a little indexes to be 
merged into the master index? Either way is fine with me but which would be 
better for Lucene?

I've also read about Remote Parallel Multi Searcher, seems I might be able 
to keep the indexes on the local hosts and use Remote Parallel MultiSearcher 
to search, but the security of the files and their content is a big issue. I 
can't be opening any back doors (ports) to the files or their contents, port 
8080 only w/HTTPS.

So I guess I am asking two questions. Is searching distributed indexes via 
Remote Parallel MultiSearcher easily doable, safe and recommended (Can RMI 
tunnel thru 8080?)  or,  given the security constraints,  is maintaining a 
master index the  better/only  way.

Going with the master index approach, would it be better/faster to index a 
new document directly into the master index or index locally and then merge 
a tiny index into the master index?

Thanks to any and all that take the time to advice me.

jim s.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message