lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Goldenberg" <>
Subject RE: Distributed Lucene.. - clustering as a requirement
Date Thu, 06 Apr 2006 19:30:27 GMT
I firmly believe that clustering support should be a part of Lucene.  We've tried implementing
it ourselves and so far have been unsuccessful.  We tried storing Lucene indices in a database
that is the back-end repository for our app in a clustered environment and could not overcome
the indexing exceptions in our custom Directory implementation.
I think it'd be perfect if some of the Lucene gurus were to implement an RDBMS-backed Directory
and post it (in addition to the sleepycat.db package that's currently in contrib).  The nitty-gritties
of dealing with Lucene indexing structures at a single byte level are just way too much trouble
to deal with for application integrator like myself.
- Dmitry


From: Samuru Jackson []
Sent: Mon 3/6/2006 10:05 AM
Subject: Re: Distributed Lucene..

Do you plan to release some kind of a commerical product including an API?

I ask because I'm evaluating different technologies for a prototype
which is part of my diploma thesis.

The problem is that I have to deal with real huge data amounts and one
machine is simply not enough to handle those amounts of data.

Lucene seems to be a good choice but it won't scale up for real big
data amounts. So I thought about expanding the indexes over several
machines in junks so that it fits into the memory of those machines.

One machine should collect the results and calculate some kind of
score out of the delivered hits from the machines.

As I'm not familiar with the concrete mechanisms of Lucene this is
just a naive thought, but I think that such a clustering mechanism
could become a killer app.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message