Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of haipengdu@gmail.com designates
 64.233.162.201 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=WZf53/Aa8gsUm0SzPWr0DUrxKZtv9FA+DQtpmzEUUvozOG4YMxrfn4ImTS4QbVtkZC2amX+k+Xku+wYN/T3DJKaDBqsdEYGzRt/itPQ/zQUgLoTyzM0dP7+Z+CIMY/zXc8YBsFYJ+xjzufIhv7WJeTTHLGtuB+3kRnAFaG3IRc0=
Message-ID: <cb1a940050907124913f17d93@mail.gmail.com>
Date: Wed, 7 Sep 2005 13:49:18 -0600
From: haipeng du <haipengdu@gmail.com>
To: java-dev@lucene.apache.org
Subject: Re: limit return results
In-Reply-To: <66E70FEE-D273-4837-BB86-60DA5A24382D@ehatchersolutions.com>
Mime-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_2084_8897528.1126122558770"
References: <D113318BC373924285F75C8AC010EC72216C7B@SHERWOOD.open.ac.uk>
	 <cb1a94005090708583a7b48a8@mail.gmail.com>
	 <66E70FEE-D273-4837-BB86-60DA5A24382D@ehatchersolutions.com>

------=_Part_2084_8897528.1126122558770
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

That is just my concern because my big number of documents I will have(at=
=20
least 4 million documents).

On 9/7/05, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>=20
> Have you seen out of memory problems? Or are you being preemptive in
> your concerns?
>=20
> Erik
>=20
> On Sep 7, 2005, at 11:58 AM, haipeng du wrote:
>=20
> > The reason that I want to limit returned result is that I do not want
> > to get out of memory problem. I index lucene with 3 million documents.
> > Sometimes, searching will return millions of fields back to me. I just
> > want to get the first 100, for example , to show them to user. Even, I
> > use search(query,filter,topDocs), I believe it still return all
> > results
> > back. So how could I limit the lucene returning?
> >
> > On 9/7/05, M.Altheim <M.Altheim@open.ac.uk> wrote:
> >
> >>
> >>
> >> Erik Hatcher [mailto:erik@ehatchersolutions.com] wrote:
> >>
> >>>
> >>> On Sep 6, 2005, at 10:47 PM, Murray Altheim wrote:
> >>>
> >>>
> >>>> Erik Hatcher wrote:
> >>>>
> >>>>
> >>>>> Just access the first 100 Hits - simple as that.
> >>>>> Erik
> >>>>>
> >>>>
> >>>> Erik,
> >>>>
> >>>> This question has come up before. For high traffic sites that
> >>>> can't afford to have the search engine accumulating thousands
> >>>> of hits, only to deliver 100, or perhaps just a few, the
> >>>> current approach *seems* like quite a lot of extra processing.
> >>>> Is there some way to have the engine simply stop generating
> >>>> the hit list after it reaches the specified threshold?
> >>>>
> >>>
> >>> The operator word here is "seems". Do you have any evidence that
> >>> doing a basic .search(Query) and only getting the first 100 results
> >>> is too slow?
> >>>
> >>> The HitCollector option that Otis mentioned is one alternative,
> >>> though I don't think it'll be much, if any, faster.
> >>>
> >>
> >> Erik,
> >>
> >> Evidence, no. I'm looking at this from the perspective of the
> >> Open University, where we have over 200,000 students accessing
> >> and searching our online services. Anything that can minimize
> >> the impact on our processors is going to be most welcome, i.e.,
> >> we don't have cycles to waste. If the student is only expecting
> >> the first 10 results and the engine generates 1000, 990 of them
> >> are wasted.
> >>
> >> Murray
> >>
> >> .....................................................................
> >> .
> >> Murray Altheim http://www.altheim.com/murray/
> >> Strategic & Service Development
> >> The Open University Library
> >> Milton Keynes, Bucks, MK7 6AA, UK
> >>
> >> Ils ont l'orteil de Bouc, & d'un Chevreil l'oreille,
> >> La corne d'un Chamois, & la face vermeille
> >> Comme un rouge Croissant: & dancent toute nuict
> >> Dedans un carrefour, ou pres d'une eau qui bruict.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >>
> >
> >
> > --
> > Haipeng Du
> > Software Engineer
> > Comphealth,
> > Salt Lake City
> >
>=20
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>=20
>=20


--=20
Haipeng Du
Software Engineer
Comphealth,=20
Salt Lake City

------=_Part_2084_8897528.1126122558770--