Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
From: Alan Woodward <alan@flax.co.uk>
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_CFD59DA1-3E19-43D1-A5E2-1676248A0022"
Subject: Re: Memory problem with TermQuery
Date: Mon, 8 Jun 2015 11:55:51 +0100
In-Reply-To: <26e6b9593bfd4e21b81d3d50ecc242bf@KIM02.tis.local>
To: java-user@lucene.apache.org
References: <87a2d61a59354b78a08e73cf35c9a53a@KIM02.tis.local>
 <144E8296-C46E-4053-A709-0E537404E7F7@flax.co.uk>
 <26e6b9593bfd4e21b81d3d50ecc242bf@KIM02.tis.local>
Message-Id: <77870F57-E8BC-44EF-95CD-7EA34C213C77@flax.co.uk>

--Apple-Mail=_CFD59DA1-3E19-43D1-A5E2-1676248A0022
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

You'll still need to call rewrite, but it needs to be done per-reader, =
so you'll need to cache the queries *before* they're rewritten, and then =
call rewrite whenever you create a new IndexReader.  Otherwise you'll =
get incorrect scores, and possibly missed hits as well.

Alan Woodward
www.flax.co.uk


On 8 Jun 2015, at 11:46, Anna Maier wrote:

> Hi Alan,
>=20
> you are right, we are calling rewrite on our query at some point. Ok, =
it would probably be an option to take that out.
> Thanks for the hint!
>=20
> Best,
> Anna
>=20
> -----Original Message-----
> From: Alan Woodward [mailto:alan@flax.co.uk]=20
> Sent: Montag, 8. Juni 2015 12:23
> To: java-user@lucene.apache.org
> Subject: Re: Memory problem with TermQuery
>=20
> Hi Anna,
>=20
> In normal usage, perReaderTermState will be null, and TermQuery will =
be very lightweight.  It's in particular expert use cases (generally =
after queries have been rewritten against a specific IndexReader) that =
the perReaderTermState will be initialized.  Are you cacheing rewritten =
queries somehow?
>=20
> Alan Woodward
> www.flax.co.uk
>=20
>=20
> On 8 Jun 2015, at 10:49, Anna Maier wrote:
>=20
>> Hi,
>>=20
>> we ran into a memory problem with TermQuery: in our program, we build =
a TermQuery object from the user input and pass it around, to be able to =
different things, like execute the query again and so on. So, the =
TermQuery object can potentially exist for some time.
>> Now it turns out, that a TermQuery keeps a reference to an =
IndexReader (via the perReaderTermState field).=20
>> This keeps our program from throwing old readers away when new ones =
are opened. This has quite an impact on the required memory, especially =
for big indices. It is not feasible anymore now to keep a reference to a =
TermQuery for longer.
>>=20
>> I'm wondering: is this a bug? After all, I would have expected the =
TermQuery to be a lightweight object. Or is the TermQuery not intended =
to be passed around in the program at all?=20
>>=20
>> Best,
>> Anna
>>=20
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>=20
>=20
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>=20


--Apple-Mail=_CFD59DA1-3E19-43D1-A5E2-1676248A0022--