Mailing-List: contact oak-dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: oak-dev@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of mueller@adobe.com designates
 64.18.1.23 as permitted sender)
From: Thomas Mueller <mueller@adobe.com>
To: "oak-dev@jackrabbit.apache.org" <oak-dev@jackrabbit.apache.org>
Date: Mon, 28 Oct 2013 07:13:46 +0000
Subject: Re: Reduce number of calls from Oak to Mongo DB on restarts
Thread-Topic: Reduce number of calls from Oak to Mongo DB on restarts
Thread-Index: Ac7TrTqAZVJ70Z2ySN2FL5zaEEBdLg==
Message-ID: <CE93CA49.4BBFE%mueller@adobe.com>
In-Reply-To: 
 <CAOFYJNbegvf4ChkzxS83haC+p+VEAM+bW6OYQg1T_HjhnzcKEg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.3.8.130913
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi,

>The SegmentMK addresses this issue by grouping related content into
>larger segments and thus reducing the number of network or disk
>accesses needed in most use cases. Perhaps the MongoMK could do
>something similar?

Yes. What might work is to store a limited number of child nodes (a
subtree really) in the parent node (in the parent document), for a certain
revision. Kind of a cache, which is only used if there is no change below
the given subtree. The subtree cache would be built in a background
thread, and outdated subtree caches could be removed whenever a document
is changed (possibly also in a background thread).

Using MapDB [1] or the H2 MVStore [2] for caching: this might improve
startup time drastically, if we really load many nodes at startup. But I'm
a bit afraid of the added complexity to maintain a local storage. So far
we don't use local storage as far as I know.

[1] http://www.mapdb.org/
[2] http://h2database.com/html/mvstore.html


Regards,
Thomas