lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben West <bwsithspaw...@yahoo.com>
Subject Re: [Lucene.Net] Server farm sharing Lucene
Date Fri, 20 May 2011 18:44:24 GMT
The idea of a scheduled task was just a very simple one. I think Microsoft's DFS is a glorified
form of this: it just listens for changes on one server and copies them over to the others.
I'm sure you can find many other tools which do something similar. You would need to check
IndexReader.IsCurrent periodically, but I guess you must already be doing that.

Also: premature optimization is the root of all evil. If you don't have any problems with
how it works now, don't let me confuse you into creating some :-) For a smallish index, all
but the most egregious misuses of Lucene are still pretty fast. 

-Ben


----- Original Message -----
From: Brian Sayatovic <bsayatovic@creditinfonet.com>
To: "lucene-net-user@lucene.apache.org" <lucene-net-user@lucene.apache.org>; Ben West
<bwsithspawn00@yahoo.com>
Cc: 
Sent: Friday, May 20, 2011 1:28 PM
Subject: RE: [Lucene.Net] Server farm sharing Lucene

I'm also concerned with the "liveliness".  We have index updates happening in conjunction
with writes to our database.  Thus, if a user creates a record, it's instantly indexed. 
That means they can create an entry and instantly search for it.

If I were to schedule period index updates, they wouldn't' be able to do this.

Thus far, our dozens of developers have been all sharing a network accessible index in this
manner.  No one has complained, but then again, we're not yet focusing on performance of
search (many other concerns in front of that).

Based on your statements, I may need to re-prioritize the risk mitigation.

Regards,
Brian.

-----Original Message-----
From: Ben West [mailto:bwsithspawn00@yahoo.com]
Sent: Friday, May 20, 2011 2:07 PM
To: lucene-net-user@lucene.apache.org
Subject: Re: [Lucene.Net] Server farm sharing Lucene

The Lucene FAQ (http://wiki.apache.org/lucene-java/ImproveSearchingSpeed) specifically warns
against using remote file systems. Depending on what you mean by "network-accessible", it
could be a lot slower. You (probably) want something of the form: the data is stored locally,
but is updated periodically from a remote location. The simplest thing is a scheduled task
which just copies over the new index every day at midnight.

Even with an ideal filesystem, you're going to have to deal with paying an additional warmup
penalty that you wouldn't get in an NRT configuration.

Another thing to note is that, while it's very easy to have multiple readers, it is really
hard to have multiple IndexWriters. We just have one writer, and deal with the fact that it's
not highly available.

Hope this helps,
-Ben

----- Original Message -----
From: Brian Sayatovic <bsayatovic@creditinfonet.com>
To: "lucene-net-user@lucene.apache.org" <lucene-net-user@lucene.apache.org>
Cc:
Sent: Friday, May 20, 2011 12:40 PM
Subject: [Lucene.Net] Server farm sharing Lucene

How have folks gone about setting up Lucene in a server farm?  Just a network-accessible
shared directory?

Regards,
Brian.
________________________________

Learn more about the products, services and technology solutions available from CIN Legal
Data Services at: www.cinlegal.com<http://www.cinlegal.com>

This message may contain confidential / proprietary information from CIN Legal Data Service
and Credit Infonet, Inc.. If you are not an intended recipient, please refrain from the disclosure,
copying, distribution or use of this information. All such unauthorized actions are strictly
prohibited. If you have received this transmission in error, please notify the sender by e-mail
at bsayatovic@creditinfonet.com and delete all copies of this material from any computer.

________________________________

Learn more about the products, services and technology solutions available from CIN Legal
Data Services at: www.cinlegal.com<http://www.cinlegal.com>

This message may contain confidential / proprietary information from CIN Legal Data Service
and Credit Infonet, Inc.. If you are not an intended recipient, please refrain from the disclosure,
copying, distribution or use of this information. All such unauthorized actions are strictly
prohibited. If you have received this transmission in error, please notify the sender by e-mail
at bsayatovic@creditinfonet.com and delete all copies of this material from any computer.


Mime
View raw message