From dev-return-15155-apmail-jackrabbit-dev-archive=jackrabbit.apache.org@jackrabbit.apache.org Tue Nov 20 18:22:08 2007 Return-Path: Delivered-To: apmail-jackrabbit-dev-archive@www.apache.org Received: (qmail 1122 invoked from network); 20 Nov 2007 18:22:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 20 Nov 2007 18:22:08 -0000 Received: (qmail 46336 invoked by uid 500); 20 Nov 2007 18:21:54 -0000 Delivered-To: apmail-jackrabbit-dev-archive@jackrabbit.apache.org Received: (qmail 46109 invoked by uid 500); 20 Nov 2007 18:21:53 -0000 Mailing-List: contact dev-help@jackrabbit.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jackrabbit.apache.org Delivered-To: mailing list dev@jackrabbit.apache.org Received: (qmail 46100 invoked by uid 99); 20 Nov 2007 18:21:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Nov 2007 10:21:53 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Nov 2007 18:22:04 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 1CBC371420B for ; Tue, 20 Nov 2007 10:21:43 -0800 (PST) Message-ID: <19081159.1195582903083.JavaMail.jira@brutus> Date: Tue, 20 Nov 2007 10:21:43 -0800 (PST) From: "Ard Schrijvers (JIRA)" To: dev@jackrabbit.apache.org Subject: [jira] Commented: (JCR-1213) UUIDDocId cache does not work properly because of weakReferences in combination with new instance for combined indexreader In-Reply-To: <4587580.1194865070552.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/JCR-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543985 ] Ard Schrijvers commented on JCR-1213: ------------------------------------- Follow up: I have been trying to cache docNumbers with respect to their IndexSegments, which obviously do change less frequently. Caching based on an entire CachingMultiReader is trivial, but also, the performance gain is to small, since the multiReader changes to frequently. But, I am having difficulties somebody might be able to help me with Bottom line is, we do not want to cache based on the entire CachingMultiReader, but on its segments it consists of. Now, I build (hacked for the time being) something that in DocId.UUIDDocId I keep track of the segment reference through a WeakReference. But....as I didn't see behavior I was expecting, I found another difficulty: The multiReader which is created in MultiIndex is a CachingMultiReader consisting of ReadOnlyIndexReader's. When something changed in one of the indexes, a new multiReader is constructed....but instead of reusing the non-changed ReadOnlyIndexReader instances, every ReadOnlyIndexReader is re-constructed (not the shared/caching reader they consist of though ) , but, since the instances our multiReader consists of are recreated, my WeakReferences based on segment instances are useless. So, instead of using a WeakReference on the multiReader segments, I could get the sharedReader instance out of it, but this means casting and adding methods, something we really do not want of course (and i am not sure if the BitSet keeing track of deleted might have changed without the sharedReader being changed ( to be honest, I cannot yet grasp the big picture about keeping track of the deleted bitset ) ). So, does anybody have an idea how we might be able to have in MultiIndex.getIndexReader() only new instances of the ReadOnlyIndexReaders which actually changed...Or is this not an option? I did try to add it to the AbstractIndex to check wether there was already an instance of ReadOnlyIndexReader but then I get AlreadyClosedException in lucene. I can make another JIRA issue for it if others think it might be valuable and not part of this issue. WDOT? > UUIDDocId cache does not work properly because of weakReferences in combination with new instance for combined indexreader > --------------------------------------------------------------------------------------------------------------------------- > > Key: JCR-1213 > URL: https://issues.apache.org/jira/browse/JCR-1213 > Project: Jackrabbit > Issue Type: Improvement > Components: query > Affects Versions: 1.3.3 > Reporter: Ard Schrijvers > Fix For: 1.4 > > > Queries that use ChildAxisQuery or DescendantSelfAxisQuery make use of getParent() functions to know wether the parents are correct and if the result is allowed. The getParent() is called recursively for every hit, and can become very expensive. Hence, in DocId.UUIDDocId, the parents are cached. > Currently, docId.UUIDDocId's are cached by having a WeakRefence to the CombinedIndexReader, but, this CombinedIndexReader is recreated all the time, implying that a gc() is allowed to remove the 'expensive' cache. > A much better solution is to not have a weakReference to the CombinedIndexReader, but to a reference of each indexreader segment. This means, that in getParent(int n) in SearchIndex the return > return id.getDocumentNumber(this) needs to be replaced by return id.getDocumentNumber(subReaders[i]); and something similar in CachingMultiReader. > That is all. Obviously, when a node/property is added/removed/changed, some parts of the cached DocId.UUIDDocId will be invalid, but mainly small indexes are updated frequently, which obviously are less expensive to recompute. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.