lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tarandeep Singh <tarand...@gmail.com>
Subject Re: Using lucene in a clustered app server
Date Thu, 11 Jun 2009 16:29:49 GMT
Hi Amin,

I am also working on similar implementation. But right now I am working on
building Lucene Index in parallel using Hadoop.

How are you building your Lucene index (or indexes) ? Have you tried Hadoop?
If yes, we can share some design issues.

Also, for serving Lucene indexes over cluster, you might be interested in
Katta project-
katta.sourceforge.net

-Tarandeep

On Thu, Jun 11, 2009 at 1:04 AM, Amin Mohammed-Coleman <aminmc@gmail.com>wrote:

> Hi
>
> we are thinking of implementing a solution like this.  All nodes on the
> cluster has a local index from which searches are performed.  Once an
> update
> on the index needs to happen the work is sent to a queue where there is a
> master who is responsible for updating the central index.  The master is
> then responsible for propagating/notifying other nodes in the cluster that
> their local index is not in sync.  The local index worker can perform an
> update against the master index.
>
> We haven't implemented this yet but its the approach we'll take inorder to
> deal with clustering.  If anyone has any ideas which could improve this or
> a
> better approach your thoughts will be appreciated.
>
> Cheers
> Amin
>
> On Wed, Jun 10, 2009 at 10:15 AM, Ian Lea <ian.lea@gmail.com> wrote:
>
> > I'd recommend using your favourite queueing service to pass all
> > updates to a central process, the one and only process that updates
> > the index.  If you don't already have a favourite queueing service,
> >
> http://en.wikipedia.org/wiki/Java_Message_Service#Provider_implementations
> > lists several JMS implementations.
> >
> >
> > --
> > Ian.
> >
> > On Tue, Jun 9, 2009 at 9:02 PM, Newman, Billy<Billy.Newman@itt.com>
> wrote:
> > > I am trying to figure out the best way to add to a lucene index across
> a
> > clustered app server.  I cannot grab an IndexWriter for each node in the
> > cluster, because I would run into lock file problems.  I am not sure if I
> > can share one IndexWriter across the cluster because what happens when
> two
> > or more nodes update the same thing.  Anyone have any insight?
> > >
> > > Thanks.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message