lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Lucene directory copy - master copy to local index
Date Sun, 21 Jun 2009 03:32:11 GMT

Hi Amin,

Have a look at Solr, it may be what you are after: http://lucene.apache.org/solr/
 
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Amin Mohammed-Coleman <aminmc@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Saturday, June 20, 2009 1:32:18 PM
> Subject: Lucene directory copy - master copy to local index
> 
> Hi
> I am prototyping the following situation:
> 
> 1) Multiple nodes in a cluster
> 2) Each node has a local index
> 3) Search requests are maded against the local index
> 4) Index updates are sent to a JMS where a master process adds document to
> index
> 5) Each node is configured to check whether the local index is out of date
> and needs to copy from master index.
> 
> I have created a class that will be configured (quartz job) to copy master
> directory files to the local directory.  I was wondering if I could get
> advice on whether this is the best way to copy master directory changes to
> the local.
> 
> public class LocalIndexProvider {
> 
> private Directory masterDirectory;
> 
> private Directory localDirectory;
> 
> private final Logger LOGGER = Logger.getLogger(getClass());
> 
> public LocalIndexProvider(Directory masterDirectory, Directory
> localDirectory) {
> 
> this.masterDirectory  = masterDirectory;
> 
> this.localDirectory = localDirectory;
> 
> }
> 
> public void updateLocalIndex() throws Exception {
> 
> StopWatch stopWatch = new StopWatch("copy-time");
> 
> stopWatch.start();
> 
> String[] masterFiles = masterDirectory.list();
> 
> String[] localFiles = localDirectory.list();
> 
> Lock lock = localDirectory.getLockFactory().makeLock("test");
> 
> lock.obtain();
> 
> try {
> 
> if (localFiles.length != masterFiles.length) {
> 
> Directory.copy(masterDirectory, localDirectory, false);
> 
> }
> 
> }finally {
> 
> lock.release();
> 
> }
> 
> stopWatch.stop();
> 
> LOGGER.debug("total time taken :" + stopWatch.getTotalTimeMillis());
> 
> }
> 
> }
> 
> 
> 
> The above is just a proof of concept and I am open to any comments about it.
> It may be inefficient or it may be.  I have looked at the source code from
> hibernate search and it seems as though they copy each file from the
> directory (master to local).  I'm not sure whether this is something I need
> to do or the above (with more refinement) would be enough.
> 
> Any help would be highly appreciated.
> 
> Cheers
> Amin


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message