Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: "Seidel. Robert" <Robert.Seidel@aeb.de>
To: "users@jackrabbit.apache.org" <users@jackrabbit.apache.org>
Date: Wed, 5 Jan 2011 12:44:23 +0100
Subject: AW: Performance issue when removing high amount of nodes
Thread-Topic: Performance issue when removing high amount of nodes
Thread-Index: AcusyD74BKHpCL3xQHWHjosyRsSKMAABQ7Ag
Message-ID: <C95151191D741844A776DE94864A68AA10A0292488@S-HQMX7.pmbelz.de>
References: <1294219451813-3175050.post@n4.nabble.com>
In-Reply-To: <1294219451813-3175050.post@n4.nabble.com>
Accept-Language: de-DE
Content-Language: de-DE
acceptlanguage: de-DE
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi,

Jackrabbit keeps all children of a node in memory, so you should not have m=
ore than 10.000 direct child nodes at a node. (See http://jackrabbit.510166=
.n4.nabble.com/Suggestions-for-node-hierarchy-td2966757.html)

The solution in your case would be to add another level like

Project1
|_Stream1
  |__Records1-10000
     |_Record 1
     |_Record 2
  |__Records10001-20000
  ...
...
|_Stream2
  |__Records1-10000
     |_Record1
     |_Record2
  ...
  |__Records110001-120000

Regards, Robert

-----Urspr=FCngliche Nachricht-----
Von: docxa [mailto:dev@docxa.com]=20
Gesendet: Mittwoch, 5. Januar 2011 10:24
An: users@jackrabbit.apache.org
Betreff: Performance issue when removing high amount of nodes


Hi,

We have to store in our repository a high amount of data, using this kind o=
f
tree:

Project1
|_Stream1
  |__Record1
  |__Record2
  ...
  |__Record120000
...
|_Stream2
  |__Record1
  |__Record2
  ...
  |__Record120000

etc.

It takes some time to add those records, which was expected, but it's even
more time-consuming to remove them. (sometimes even crashing the VM)
I understand it has to do with Jackrabbit putting it all in memory to check
for referential integrity violations.

While searching for answers on the mailing list I saw two ways of dealing
with this:
1- Deactivate referential integrity checking. I tried that, and it did not
seem to accelerate the process, so I may be doing it wrong. (And I guess
it's quite wrong to even do it)
2- Recursively removing nodes by packs.

I noticed than when using the second method, the more children a node have,
the more time it will take to remove some of them. So I guess it would be
best to try and split the records through multiple subtrees.

So I'd like to know if there is a better way of organizing my data in order
to improve the adding and removing operations. And if the deactivation of
referential integrity checking is really risky, and how I'm supposed to do
it? (I tried subclassing RepositoryImpl and using
setReferentialIntegrityChecking but it didn't seem to change anything)

Thank you for your help.

A. Mariette
DOCXA
--=20
View this message in context: http://jackrabbit.510166.n4.nabble.com/Perfor=
mance-issue-when-removing-high-amount-of-nodes-tp3175050p3175050.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.