Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@jackrabbit.apache.org
Received-SPF: pass (herse.apache.org: domain of cris.daniluk@gmail.com
 designates 66.249.92.175 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=XMahZ/4OQ4vbJOUW1z7aIDO9737BEs51wLTWOQYiW6cuj9T2SNYZak1JfgThoErIkA02VNJ/+IspjpslcZlyQ237fbFmdgdh8qa+N8vv1+jo/f0haNOsUy+/wbbah9JQdlfOSsPC1mOc448jZb9XYEo5ne57eNJ3/Fl302vMYHI=
Message-ID: <ac19d2a60705230811o1e54ab57hca3d419014976115@mail.gmail.com>
Date: Wed, 23 May 2007 11:11:47 -0400
From: "Cris Daniluk" <cris.daniluk@gmail.com>
To: users@jackrabbit.apache.org
Subject: Re: workspace / repository scalability
In-Reply-To: <eb7e2190705230755q4dfd4ad9pa20925d6ce47bfff@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <ac19d2a60705220648x78cd79d2u4a6738837ddf22e7@mail.gmail.com>
	 <eb7e2190705230726l656c522fgf2aff037166d280f@mail.gmail.com>
	 <ac19d2a60705230732k5c858cb4r521ff7de8e1db5c1@mail.gmail.com>
	 <eb7e2190705230755q4dfd4ad9pa20925d6ce47bfff@mail.gmail.com>

> > When you say adequate hierarchical structure, does this imply that we should
> > try to keep our tree "bushy"? Really, because we rely on the external search
> > engine for location, we only direct query on sequential ID at the database.
> > Should a partitioning strategy be used? If so, what sort of depth might we
> > aim for?
> i see... i think it is important to mention that jackrabbit is not optimized for
> long lists of child nodes currently, so i would recommend to stay away if
> possible from more than a couple of hundred child nodes.
> as a guidance for hierarchy i usually use something like:
> "if i wouldn't do it in a filesystem, i don't do it in a content repository"
> (assuming that i view a node as a file or folder)
>
> so let's assume your sequential hex-id is something like "123abc" i would
> recommend something like a partitioning for the node structure as follows:
> /12/3a/bc which leaves you with 256 child nodes per node.
>

So some form of hashing sounds desirable. If the hashed nodes were
mapped to a separate node structure / workspace that did not hash to
create a logical view, would the performance impact still apply?

> personally, i like to use the derby persistence manager with external
> fs based blobs (standard setup). with this setup i do
> "hot backups" by just backing up the full repository folder in the filesystem.


GBs of data in derby of course makes me nervous on a probably
irrational level. The fs-based blobs is interesting, and may make DR
replication easier. You mentioned some testing... it would be great
for us to do similar testing with mock data reflecting our
environment. If the tests you performed included any special
harnesses, configuration, etc, would it be possible to see them?

Thanks for all your help!

Cris