lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay B <vijay.nip...@gmail.com>
Subject Index replication strategy
Date Thu, 04 Dec 2014 14:19:05 GMT
Hello,

We index docs coming from database nightly. Current index is sitting on
NFS. Due to obvious performance reasons, we are switching are planning to
switch to local index. W have cluster of 4 servers and with NFS it was not
a problem for us until now to share the index. but going forward, we are
looking for our design options for index replication on to local storage.

Our setup:
Index size: 8GB (grows by 2GB every year)
Lucene: 4.2.1
64-bit Java

The options we considered:
 *  Each server instance,hosting a nightly job to pull delta of data from
db. But, this would result in high DB load. (4 severs =4 times the load)

 * An additional nightly job sitting on another sever, that pushes the data
on to local disks of each instances..This may not work out as the local
disk may not be visible.

* each sever hosting a replication job that pulls delta of data from NFS
and stores in the local index...so far this is the only promising option we
have.

* Does solr an option for us in this case? (I know it's a question for solr
group..but experts here might have some thoughts..)..

Thank you for your attention.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message