Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of vfunstein@gmail.com
 designates 209.85.212.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <01AFE0FB733B9944974A82A09CEB7A0309C835D36D@mail3.imedx.com>
References: <01AFE0FB733B9944974A82A09CEB7A0309C81ABB21@mail3.imedx.com>
	<1400570841.2420.155.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D126@mail3.imedx.com>
	<1400578244.2420.170.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D16A@mail3.imedx.com>
	<1400581087.2420.182.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D36D@mail3.imedx.com>
Date: Fri, 23 May 2014 16:52:22 -0700
Message-ID: 
 <CALr4HzpTE1otyEhB+FNk=LjtTjnLOWeh6Gy7fMcAeSGzs9_6sQ@mail.gmail.com>
Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server
From: Vitaly Funstein <vfunstein@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=e89a8f502eeeb657c104fa19eb88

--e89a8f502eeeb657c104fa19eb88
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

At the risk of sounding overly critical here, I would say you need to scrap
your entire approach of building one small index per request, and just
build your entire searchable data store in Lucene/Solr. This is the
simplest and probably most maintainable and scalable solution. Even if your
index contains 10M+ documents, returning at most 500 search results should
be lightning fast compared to the latencies you're seeing right now. To
facilitate data export from the DB, take a look at this:
http://wiki.apache.org/solr/DataImportHandler


On Tue, May 20, 2014 at 7:36 AM, Shruthi <ssethi@imedx.com> wrote:

>
>
>
>
> -----Original Message-----
> From: Toke Eskildsen [mailto:te@statsbiblioteket.dk]
> Sent: Tuesday, May 20, 2014 3:48 PM
> To: java-user@lucene.apache.org
> Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server
>
> On Tue, 2014-05-20 at 11:56 +0200, Shruthi wrote:
>
> Toke:
> > Is 20 second an acceptable response time for your users?
> >
> > Shruthi: Its definitely not acceptable. PFA the piece of code that we
> > are using..Its taking 20seconds. That=E2=80=99s why I drafted this tick=
et to
> > see where I was going wrong.
>
> Indexing 1000 documents/sec in Lucene is quite common, so even taking
> into account large documents, 20 seconds sounds like quite a bit.
> Shruthi: I had attached the code snippet in previous mail. Do you suspect
> a foul play there?
>
> > Shruthi: Well,  its two stage process: Client is looking at
> > historical data based on a parameters like names, dates,MRN, fields
> > etc.. SO the query actually gets the data set fulfilling the
> > requirements
> >
> > If client is interested in doing a text search then he would pass the
> > search phrase on the result set.
>
> So it is not possible for a client to perform a broad phrase search to
> start with. And it sounds like your DB-queries are all simple matching?
> No complex joins and such? If so, this calls even more for a full
> Lucene-index solution, which handles all aspect of the search process.
> Shruthi: We call a DB stored procedure to get us the result set for
> working with..
> We will be using highlighter API and  I don=E2=80=99t think Memory  index=
 can be
> used with highlighter.
>
> >
> - Toke Eskildsen, State and University Library, Denmark
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--e89a8f502eeeb657c104fa19eb88--