lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From testn <>
Subject Re: Clustered Indexing on common network filesystem
Date Thu, 02 Aug 2007 15:12:38 GMT

Why don't you check out Hadoop and Nutch? It should provide what you are
looking for.

Zach Bailey wrote:
> Hi,
> It's been a couple of days now and I haven't heard anything on this 
> topic, while there has been substantial list traffic otherwise.
> Am I asking in the wrong place? Was I unclear?
> I know there are people out there that have used/are using Lucene in a 
> clustered environment. I am just looking for any sort of feedback 
> (general or specific) about clustering lucene as well as filesystem 
> compatibility (windows shares, NFS, etc.).
> Thanks again,
> -Zach
> Zach Bailey wrote:
>> Hello all,
>> First a little background - we are developing a clustered application 
>> that will in part leverage Lucene to provide index and search 
>> capabilities. We have already spent time investigating various index 
>> storage implementations (database vs. filesystem) and we've decided for 
>> performance reasons to go with a filesystem index storage scheme.
>> That said, I have read back through the archives a bit and noticed that 
>> the support for index storage on NFS is still experimental (e.g. the 
>> latest bugfixes have not made it out to an official, stable release). I 
>> realize most of the issues related to using a shared file system revolve 
>> around locking, and I haven't seen much about the maturity of locking 
>> for other network filesystems.
>> I was wondering if anyone has tried any other networked filesystems or 
>> had any recommendations. We have clients who would be doing this on both 
>> Windows and Unix/Linux so any insight there would be appreciated as well 
>> - it can be assumed that across any cluster the operating system use 
>> would be homogeneous (i.e. all nodes are on windows and would use 
>> windows shares, or all nodes are on linux and would use xyz filesystem).
>> Thanks in advance,
>> -Zach Bailey
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View this message in context:
Sent from the Lucene - Java Users mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message