lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay B <vijay.nip...@gmail.com>
Subject Re: Index replication strategy
Date Thu, 04 Dec 2014 15:29:52 GMT
Looks very promising. I will check out your blog.

On Thu, Dec 4, 2014 at 9:38 AM, Shai Erera <serera@gmail.com> wrote:

> Ooops, didn't notice that :).
>
> So you'll need to upgrade to Lucene 4.4.0 in order to use it. You can read
> some details as well as example code here:
> http://shaierera.blogspot.com/2013/05/the-replicator.html.
>
> Shai
>
> On Thu, Dec 4, 2014 at 4:36 PM, Vijay B <vijay.nipuna@gmail.com> wrote:
>
> > As indicated in my post, we use Lucene 4.2.1.
> >
> > On Thu, Dec 4, 2014 at 9:29 AM, Shai Erera <serera@gmail.com> wrote:
> >
> > > Do you use Lucene or Solr? Lucene also has a replication module, which
> > will
> > > allow you to replicate index changes.
> > >
> > > On Thu, Dec 4, 2014 at 4:19 PM, Vijay B <vijay.nipuna@gmail.com>
> wrote:
> > >
> > > > Hello,
> > > >
> > > > We index docs coming from database nightly. Current index is sitting
> on
> > > > NFS. Due to obvious performance reasons, we are switching are
> planning
> > to
> > > > switch to local index. W have cluster of 4 servers and with NFS it
> was
> > > not
> > > > a problem for us until now to share the index. but going forward, we
> > are
> > > > looking for our design options for index replication on to local
> > storage.
> > > >
> > > > Our setup:
> > > > Index size: 8GB (grows by 2GB every year)
> > > > Lucene: 4.2.1
> > > > 64-bit Java
> > > >
> > > > The options we considered:
> > > >  *  Each server instance,hosting a nightly job to pull delta of data
> > from
> > > > db. But, this would result in high DB load. (4 severs =4 times the
> > load)
> > > >
> > > >  * An additional nightly job sitting on another sever, that pushes
> the
> > > data
> > > > on to local disks of each instances..This may not work out as the
> local
> > > > disk may not be visible.
> > > >
> > > > * each sever hosting a replication job that pulls delta of data from
> > NFS
> > > > and stores in the local index...so far this is the only promising
> > option
> > > we
> > > > have.
> > > >
> > > > * Does solr an option for us in this case? (I know it's a question
> for
> > > solr
> > > > group..but experts here might have some thoughts..)..
> > > >
> > > > Thank you for your attention.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message