lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitaly Funstein <vfunst...@gmail.com>
Subject Re: NewBie To Lucene || Perfect configuration on a 64 bit server
Date Fri, 23 May 2014 23:52:22 GMT
At the risk of sounding overly critical here, I would say you need to scrap
your entire approach of building one small index per request, and just
build your entire searchable data store in Lucene/Solr. This is the
simplest and probably most maintainable and scalable solution. Even if your
index contains 10M+ documents, returning at most 500 search results should
be lightning fast compared to the latencies you're seeing right now. To
facilitate data export from the DB, take a look at this:
http://wiki.apache.org/solr/DataImportHandler


On Tue, May 20, 2014 at 7:36 AM, Shruthi <ssethi@imedx.com> wrote:

>
>
>
>
> -----Original Message-----
> From: Toke Eskildsen [mailto:te@statsbiblioteket.dk]
> Sent: Tuesday, May 20, 2014 3:48 PM
> To: java-user@lucene.apache.org
> Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server
>
> On Tue, 2014-05-20 at 11:56 +0200, Shruthi wrote:
>
> Toke:
> > Is 20 second an acceptable response time for your users?
> >
> > Shruthi: Its definitely not acceptable. PFA the piece of code that we
> > are using..Its taking 20seconds. That’s why I drafted this ticket to
> > see where I was going wrong.
>
> Indexing 1000 documents/sec in Lucene is quite common, so even taking
> into account large documents, 20 seconds sounds like quite a bit.
> Shruthi: I had attached the code snippet in previous mail. Do you suspect
> a foul play there?
>
> > Shruthi: Well,  its two stage process: Client is looking at
> > historical data based on a parameters like names, dates,MRN, fields
> > etc.. SO the query actually gets the data set fulfilling the
> > requirements
> >
> > If client is interested in doing a text search then he would pass the
> > search phrase on the result set.
>
> So it is not possible for a client to perform a broad phrase search to
> start with. And it sounds like your DB-queries are all simple matching?
> No complex joins and such? If so, this calls even more for a full
> Lucene-index solution, which handles all aspect of the search process.
> Shruthi: We call a DB stored procedure to get us the result set for
> working with..
> We will be using highlighter API and  I don’t think Memory  index can be
> used with highlighter.
>
> >
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message