lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cedric Ho" <cedric...@gmail.com>
Subject Re: Lucene 2.3.0 and NFS
Date Thu, 10 Apr 2008 01:41:18 GMT
I think cygwin will do the trick.

On Thu, Apr 10, 2008 at 9:10 AM, Rajesh parab <rajesh_parab_1@yahoo.com> wrote:
> Hi All,
>
>  Has anyone used rsync or similar utilities on Windows
>  OS to replicate Lucene index across multiple machines?
>
>  Any pointers on it will be very useful?
>
>  Regards,
>  Rajesh
>
>
>
>  --- Rajesh parab <rajesh_parab_1@yahoo.com> wrote:
>
>  > Hi Michael,
>  >
>  > Thanks a lot for your suggestions.
>  >
>  > I was looking at rsync; as per this link
>  > (http://samba.anu.edu.au/rsync/features.html), rsync
>  > is a file transfer program for UNIX. Is there rsync
>  > support for Windows as well? I found few rsync
>  > programs that works for Windows, but I am not sure
>  > if
>  > they will server the purpose. Has anyone using rsync
>  > on Windows?
>  >
>  > Regards,
>  > Rajesh
>  >
>  > --- Michael McCandless <lucene@mikemccandless.com>
>  > wrote:
>  >
>  > >
>  > > Rajesh parab wrote:
>  > > > Hi,
>  > > >
>  > > > We are currently using Lucene 2.0 for full-text
>  > > > searches within our enterprise application,
>  > which
>  > > can
>  > > > be deployed in clustered environment. We
>  > generate
>  > > > Lucene index for data stored inside relational
>  > > > database.
>  > > >
>  > > > As Lucene 2.0 did not have solid NFS support and
>  > > as we
>  > > > wanted Lucene based searches to work properly in
>  > > > Clustered environment, we had decided on
>  > following
>  > > > approach:
>  > > > 1. The index generation happens on a machine
>  > > (could be
>  > > > one of the cluster nodes or a separate machine)
>  > > and
>  > > > once the Lucene index is generated, we copy all
>  > > the
>  > > > index files to the database.
>  > >
>  > > Note that you can do also incremental replication:
>  > > often, the Lucene
>  > > index changes in minor ways (eg a single new
>  > segment
>  > > is flushed and a
>  > > new segments_N and segments.gen is written) so you
>  > > should only sync
>  > > the files that are new (and remove the ones that
>  > are
>  > > now gone).
>  > > Lucene's write-once approach makes this very
>  > simple
>  > > (you just have to
>  > > compare file names, not the contents of each
>  > file).
>  > >
>  > > It's also possible to replicate without using a
>  > DB.
>  > > EG rsync does a
>  > > great job.
>  > >
>  > > > 2. The index search request on each cluster node
>  > > > retrieves the index files from database (during
>  > > first
>  > > > search or after index update), copies to the
>  > file
>  > > > system and use it for searches.
>  > > > 3. Thus, each cluster node has its own copy of
>  > the
>  > > > index and it keeps on picking up latest version
>  > if
>  > > it
>  > > > is available inside database.
>  > > >
>  > > > This has worked fine for us till now, though we
>  > > will
>  > > > not be able continue with this model in future
>  > as
>  > > we
>  > > > want to support Lucene based searches across our
>  > > > application and also want to index large
>  > > components
>  > > > inside our application like Wiki, forums, etc.
>  > As
>  > > the
>  > > > index will grow, storing and retrieving index
>  > > files
>  > > > from database will not be an efficient
>  > operation.
>  > > >
>  > > > My questions are:
>  > > > - Will we be able to use NFS if we move to
>  > Lucene
>  > > > 2.3.0?
>  > >
>  > > Make sure you update to 2.3.1, not 2.3.0.
>  > >
>  > > > - Will there be any significant performance
>  > impact
>  > > on
>  > > > index generation and searches if we move to NFS?
>  > > > - Is Lucene + NFS combination supported for all
>  > > > operating systems? (We support Windows, Solaris,
>  > > AIX,
>  > > > HP-UX, Red Hat Linux)
>  > >
>  > > NFS *should* work, however:
>  > >
>  > >    * It's not widely used, so, test thoroughly in
>  > > your particular setup.
>  > >
>  > >    * Most likely to work is if you use a single
>  > > machine writing to
>  > > the index, and many readers.
>  > >
>  > >    * Performance is likely not great, especially
>  > on
>  > > searching, but
>  > > you should test in your specific situation.
>  > >
>  > > > - Is there any other alternative available other
>  > > than
>  > > > NFS?
>  > > >
>  > >
>  > > It's also possible to replicate without using a
>  > DB.
>  > > EG rsync does
>  > > agreat job.
>  > >
>  > > You should look at Solr, since it already has all
>  > > the infrastructure
>  > > toaccept updates, replicate index changes to
>  > remote
>  > > machines, etc.
>  > >
>  > > > I will really appreciate your comments/thoughts
>  > on
>  > > > this topic.
>  > > >
>  > > > Regards,
>  > > > Rajesh
>  > > >
>  > > >
>  > > >
>  > > >
>  > >
>  >
>  ______________________________________________________________________
>  > >
>  > > > ______________
>  > > > You rock. That's why Blockbuster's offering you
>  > > one month of
>  > > > Blockbuster Total Access, No Cost.
>  > > >
>  > http://tc.deals.yahoo.com/tc/blockbuster/text5.com
>  > > >
>  > > >
>  > >
>  >
>  ---------------------------------------------------------------------
>  > > > To unsubscribe, e-mail:
>  > > java-user-unsubscribe@lucene.apache.org
>  > > > For additional commands, e-mail:
>  > > java-user-help@lucene.apache.org
>  > > >
>  > >
>  > >
>  > >
>  >
>  ---------------------------------------------------------------------
>  > > To unsubscribe, e-mail:
>  > > java-user-unsubscribe@lucene.apache.org
>  > > For additional commands, e-mail:
>  > > java-user-help@lucene.apache.org
>  > >
>  > >
>  >
>  >
>  >
>  >
>  >
>  ____________________________________________________________________________________
>  > You rock. That's why Blockbuster's offering you one
>  > month of Blockbuster Total Access, No Cost.
>  > http://tc.deals.yahoo.com/tc/blockbuster/text5.com
>  >
>  >
>  ---------------------------------------------------------------------
>  > To unsubscribe, e-mail:
>  > java-user-unsubscribe@lucene.apache.org
>  > For additional commands, e-mail:
>  > java-user-help@lucene.apache.org
>  >
>  >
>
>
>
> __________________________________________________
>  Do You Yahoo!?
>  Tired of spam?  Yahoo! Mail has the best spam protection around
>  http://mail.yahoo.com
>
>
>
>  ---------------------------------------------------------------------
>  To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message