lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haberl Norbert <Norbert.Hab...@ssi-schaefer.com>
Subject AW: What is Lucene.net best practice architecture in ASP.NET (Web) application(s) ?
Date Tue, 19 Sep 2017 06:20:27 GMT
Sounds good ... one more question is, do I have to release the SearchManager after every search
like done here? -> http://programagic.ca/blog/rest-api-lucenenet-part-2-a-few-tricks-and-tips
If it handles concurrency by itself why I see posts where the SearcherManager is acquired
and released ? (I've seen this recommendation a couple times)

How can I achieve that the searcher includes the last changes ? I guess a refresh on every
search request will be some sort of overhead?

Best regards,
Norbert Haberl | Software Engineer | Customer Service & Support
Phone +43 316 6096 - 8435
SSI SCHÄFER | SSI Schäfer Automation GmbH | Fischeraustraße 27 | 8051 Graz | Austria

Norbert.Haberl@ssi-schaefer.com
 
Website | Blog | YouTube | Facebook


-----Ursprüngliche Nachricht-----
Von: JA Purcell [mailto:9urc311@gmail.com] 
Gesendet: Montag, 18. September 2017 15:47
An: user@lucenenet.apache.org
Betreff: Re: What is Lucene.net best practice architecture in ASP.NET (Web) application(s)
?

I think that sounds right.  In my case, I used a separate service process with an IndexWriterFactory
that handle the lifecycle of the writer only re-opening as needed as recommended by the documentation
(i.e. out of memory exception).

Like Paul said previously, you only want to re-open a new IndexReader if the index has changed.
 As an example, the 4.08 beta release has a SearcherManager that does this (see https://github.com/apache/lucenenet/blob/62badd246b68081b2fe84275e04edf633b8255d3/src/Lucene.Net/Search/SearcherManager.cs
).

On Sun, Sep 17, 2017 at 11:11 PM Haberl Norbert < Norbert.Haberl@ssi-schaefer.com> wrote:

> So in case of ASP.NET I will implement a service layer as singleton 
> and call it when needed.
> Within that layer I can reopen the reader as new docs where indexed 
> over the writer.
>
> Any thoughts on this?
>
> Best regards,
> Norbert Haberl | Software Engineer | Customer Service & Support Phone 
> +43 316 6096 - 8435 <+43%20316%2060968435> SSI SCHÄFER | SSI Schäfer 
> Automation GmbH | Fischeraustraße 27 
> <https://maps.google.com/?q=Fischeraustra%C3%9Fe+27&entry=gmail&source
> =g>
> | 8051 Graz | Austria
>
> Norbert.Haberl@ssi-schaefer.com
>
> Website | Blog | YouTube | Facebook
>
> -----Ursprüngliche Nachricht-----
> Von: Paul Irwin [mailto:pirwin@feature23.com]
> Gesendet: Freitag, 15. September 2017 16:19
> An: user@lucenenet.apache.org
> Betreff: RE: What is Lucene.net best practice architecture in ASP.NET
> (Web) application(s) ?
>
> For 3.0.3, generally you should share (i.e. make static) an 
> IndexReader where possible, potentially checking IsCurrent to see if 
> it needs to be reopened. From the official docs:
>
> "NOTE: IndexReader instances are completely thread safe, meaning 
> multiple threads can call any of its methods, concurrently. If your 
> application requires external synchronization, you should not 
> synchronize on the IndexReader instance; use your own (non-Lucene) objects instead."
>
> As for writes, I recommend if possible having a single writer (single 
> process, single server) to avoid dealing with 
> LockObtainFailedExceptions from concurrency. Do not keep an 
> IndexWriter open longer than needed, and ensure you have proper error handling to make
sure the lock is disposed of.
> Otherwise your index will get locked and may or may not be in an 
> invalid state needing repair. If you are expecting a constant flood of 
> writes (many per second sustained) and cannot batch them then you can 
> keep it open but you do run a greater risk of the process being 
> unexpectedly terminated and your index being locked or worse. If you 
> are in that boat, commit often. If you absolutely need multiple 
> writers (I strongly question this), expect a LockObtainFailedException 
> and use something like the Transient Fault Handling Application Block 
> and/or a queue to retry. But again, best case scenario is infrequent 
> or batched writes with a single writer that is opened, committed, and disposed of as
soon as possible.
>
> Since the question is about best practices for architecture when using 
> Lucene, unless it is a trivially simple app, I would recommend having 
> some kind of service/API layer to do all of your Lucene work, and call 
> that from the web application.
>
> Paul
>
> -----Original Message-----
> From: JA Purcell [mailto:9urc311@gmail.com]
> Sent: Friday, September 15, 2017 9:32 AM
> To: user@lucenenet.apache.org
> Subject: Re: What is Lucene.net best practice architecture in ASP.NET
> (Web) application(s) ?
>
> I also have the same question, but I am currently using Lucene 3.03 
> along with an ASP.NET application.  Since the web app isn't the only 
> writer to the index, I created a service that receives messages via 
> SQL Service Broker that are to be indexed.  The service only uses one 
> instance of the writer at a time shared among many threads.
>
> I am currently using a new instance of an IndexReader per request as 
> an initial implementation, but I know the best practice is to use only 
> one instance of a reader as much as possible because of initialization costs.
>
> I'm currently experimenting with building a large index as quickly as 
> possible, and so far it seems that by using multiple processes writing 
> to sub-indexes and combining them in the end is the fastest way to get it done.
>
> -Adam
>
> On Fri, Sep 15, 2017 at 3:29 AM Haberl Norbert < 
> Norbert.Haberl@ssi-schaefer.com> wrote:
>
> > Hey,
> >
> > what is generally the best practice architecture when using Lucene 
> > within an ASP.NET web application (let's say MVC).
> > Should ONE instance of IndexReader be declared or per request? Same 
> > for writers ... shall all writes be queued?
> >
> > I guess it's a common scenario to use Lucene within a web app for 
> > 1st level tier or as a separate search index but it's not clear how 
> > to use it definetly.
> >
> > Would be great to hear what you think (I'm gonna make a blog post 
> > out of this on my site)
> >
> > Best regards,
> > Norbert
> >
> >
>
Mime
View raw message