jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ard Schrijvers <a.schrijv...@onehippo.com>
Subject Re: Searching for a property
Date Fri, 18 Dec 2009 09:09:50 GMT
On Thu, Dec 17, 2009 at 10:59 PM, Dennis van der Laan
<d.g.van.der.laan@rug.nl> wrote:
> Dennis van der Laan wrote:

> See the increase of time spent on the execution: 400+ ms instead of 7ms.
> And this is not a single incident, I see this increase on all queries
> like the above.
> The memory of the JVM should not be a problem, it's set to 2Gb and only
> 800Mb is used at the moment the queries are slow. Restarting the
> application does not help either.

No, this seems logical to me. The memory is consumed by internal
lucene term enums. I am quite sure what your issue is, but did not
test it, nor ever tried it myself. But, I have always wondered *how*
the fn:lower-case could have been implemented efficiently in
Jackrabbit. It doesn't fit into my understanding of how inverted
indexes work, what Lucene is in the end. So, I am happy that my
understanding was correct, and unhappy that fn:lower-case does (again,
from top of my head and looking at code only) not scale to well.

I think in your setup a lot of time is spend in the CaseTermQuery,
which traverses all your 1 million virtualpaths first and lowercase
it. This cannot scale (nor in cpu, nor in memory).

So, would you like to give me an indication about the query execution
time without the fn:lower-case? I think it will drop to < 1 ms.

I think you should try to get away without using the fn:local-name if
this works for you. Just make sure that you store the virtualpath
property always as lower-case: then, you are fine

> Again, any help will be appreciated.

let me know if this helped,

Regards Ard

> Dennis
>>> Furthermore, of course, index size matters as well

View raw message