Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of stefan.guggisberg@gmail.com
 designates 74.125.82.50 as permitted sender)
Subject: Re: Using Jackrabbit/JCR as IDE workspace data backend
References: <63E7BD93-5385-4673-82AB-4D56A2D5CC95@gmail.com>
 <CAFYk8Nmo8FWApCZdq+5=4Vzo8ZuhpMARNzG5KiU2jeNsM5gHrQ@mail.gmail.com>
 <CAJ2P4pA85u1VCzzbNwFnPEmSKk259Cj8e-yHTRJ=YJBq2tpQkg@mail.gmail.com>
 <CAFYk8N=_KOu3+VcZC78MQ71R3+EiNW_nApPGuwR7JMb6Q1sd5g@mail.gmail.com>
 <F1CF21EC-B1BE-4DFC-B262-3CA0A83441CC@gmail.com>
From: Stefan Guggisberg <stefan.guggisberg@gmail.com>
Content-Type: text/plain;
	charset=us-ascii
In-Reply-To: <F1CF21EC-B1BE-4DFC-B262-3CA0A83441CC@gmail.com>
Message-Id: <BE38AD09-4785-4E6D-9ACF-3BD0E3FC5A8D@gmail.com>
Date: Mon, 26 Sep 2011 20:42:01 +0200
To: "users@jackrabbit.apache.org" <users@jackrabbit.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (iPhone Mail 8L1)


On 26.09.2011, at 19:56, Marcel Bruch <marcel.bruch@gmail.com> wrote:

> Hi Stefan,
>=20
> On 26.09.2011, at 18:13, Stefan Guggisberg wrote:
>=20
>>> I wrote a fairly ad-hoc dump of the 5900 data files into Jackrabbit.
>>> Storing ~240 MB took roughly 3 minutes. Is this the expected time such
>>> an operation takes? Is it possible to improve the performance somehow?
>>=20
>> the performance seems rather poor. it's hard to tell what's wrong
>> without having the test data. i noticed that you're storing the
>> content of the .json files as string properties. why aren't you
>> storing the json data as nodes & properties?
>=20
> I had no code available for serializing the data as JCR nodes. Is there an=
y simple snippet available somewhere?
> However, I thought as a first baseline this would work.=20
>=20
>=20
>> anyway, i quickly ran an adapted ad hoc test on my machine
>> (macbook pro 2.66 ghz, standard harddisk). the test imports
>> an 'svn export' of jackrabbit/trunk.
>>=20
>> importing ~6500 files takes ~30s which is IMO decent.
>=20
> Thanks for writing your test agains your local files!
>=20
> I run your code and compared the execution times. Unfortunately, it's not p=
erforming  faster :(=20
> The minute delta might be cause by some file traversing differences of by t=
he additional nodes/properties created in your code.
>=20
> However, the overall performance is still a bit low (2:24-3:05 minutes in a=
 clean repository). Any idea how the performance could be improved? Am I doi=
ng something conceptually wrong?

did you run my test with the same test data (local svn export of jackrabbit t=
runk)?

cheers
stefan

> I'm assuming that there is no big delta between creating hundreds of nodes=
 and properties compared to dumping a file's content into Jackrabbit. Is thi=
s correct?
>=20
> Thanks,
> Marcel
>=20
> =3D=3D=3D Experiments performance results =3D=3D=3D
>=20
>=20
> Jackrabbit First Hops code adapted:
>=20
> 0:00:08.522: 500 units persisted.  data 17 MB=20
> 0:00:17.057: 1000 units persisted.  data 33 MB=20
> 0:00:31.763: 1500 units persisted.  data 53 MB=20
> 0:00:41.404: 2000 units persisted.  data 72 MB=20
> 0:00:53.140: 2500 units persisted.  data 97 MB=20
> 0:01:02.988: 3000 units persisted.  data 113 MB=20
> 0:01:16.314: 3500 units persisted.  data 133 MB=20
> 0:01:35.171: 4000 units persisted.  data 143 MB=20
> 0:01:49.414: 4500 units persisted.  data 173 MB=20
> 0:02:04.617: 5000 units persisted.  data 204 MB=20
> 0:02:12.593: 5500 units persisted.  data 221 MB=20
> Mon Sep 26 19:54:58 CEST 2011: 5927 units persisted
> Run took 0:02:24.505
>=20
>=20
> Mailing List proposal:
>=20
> 0:00:14.853: 500 units persisted. data  17 MB
> 0:00:26.353: 1000 units persisted. data  33 MB
> 0:00:36.114: 1500 units persisted. data  53 MB
> 0:00:53.274: 2000 units persisted. data  72 MB
> 0:01:06.643: 2500 units persisted. data  97 MB
> 0:01:18.230: 3000 units persisted. data  113 MB
> 0:01:36.765: 3500 units persisted. data  133 MB
> 0:01:44.245: 4000 units persisted. data  143 MB
> 0:02:04.026: 4500 units persisted. data  173 MB
> 0:02:37.533: 5000 units persisted. data  204 MB
> 0:02:48.089: 5500 units persisted. data  221 MB
> Run took 0:03:08.458
>=20
>=20