Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Toke Eskildsen <te@statsbiblioteket.dk>
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
Date: Thu, 2 Oct 2014 21:41:47 +0200
Subject: RE: Lucene Java Caching Question
Thread-Topic: Lucene Java Caching Question
Thread-Index: Ac/eXx5KVezg7Lh+Q+CRL78YY9FznwAGBEvM
Message-ID: 
 <2E6A89A648463A4EBF093A9062C166830552B8A78E54@SBMAILBOX1.sb.statsbiblioteket.dk>
References: <1412267798946-4162354.post@n3.nabble.com>
In-Reply-To: <1412267798946-4162354.post@n3.nabble.com>
Accept-Language: en-US, da-DK
Content-Language: en-GB
acceptlanguage: en-US, da-DK
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

parth_n [nagarkar@asu.edu] wrote:
> I run my java application (set of spatial queries) to get the execution t=
ime
> and results for the queries. The application is terminated. Whenever I
> re-run the application with the same set of queries, the execution time i=
s
> very low comparative to the first run. So I am assuming that there is som=
e
> caching going on, but where is this stored?

It is the disk cache of your operating system. It is independent of Lucene =
and is in-memory. Most modern operating systems uses all free memory for di=
sk cache.

Lucene uses random access all the time and search speed is largely dictated=
 by how fast it can do such reads. If the data are in your disk cache, they=
 will be fetched _very_ fast.

> I have looked on for similar question on this forum, but it seems no one =
has
> come across this particular problem.

Problem? You mean for testing? Well, it is quite hard to test Lucene perfor=
mance. Related to disk cache there are three strategies:

1) Empty the disk cache before you test (how you do that depends on your op=
erating system). This makes the tests fairly repeatable, but say nothing ab=
out real world performance, as there is always some amount of caching going=
 on when you're running for real.

2) Fill the disk cache, either by repeating your test a few times and measu=
ring the last result or by reading all your index files into disk cache bef=
ore you start (on linux, 'cat * > /dev/null' should work). Again this ensur=
es test repeatability, but it is only representative of real world performa=
nce if your production index size is less than the amount of free memory.

3) Try to simulate a real setup, with some queries from your production sys=
tem, before you start your test. This is tricky to get right, but the only =
somewhat-sound approximation of real world performance.

- Toke Eskildsen

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org