jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Kurla" <stefan.ku...@gmail.com>
Subject Re: importing jackrabbit into jackrabbit
Date Wed, 25 Apr 2007 17:29:02 GMT
Hi,

Thanks for the link. I found the link yesterday and made the changes.
I ran the exact same configuration against Mysql with mediumblobs and
ran the importing system overnight. Again, we are talking about the
same number of files. The test machine that I was running on had only
512MB RAM. Maybe because of that each new file input was taking about
30 seconds on average to import into the repository, I was upto
importing 1K files when I stopped the import.

So I am now testing the importing system against a machine which has
2GB RAM. The data that is being imported is 78.6 MB on disk and has
12K file and 8K folders (the fresh SVN update from jackrabbit). The
importer is running using RMI to get to the tomcat server on the
localhost. MYSQL is also onthe localhost.

The import has been running over the last hour and I see at file
number 500 the import time has gone upto 3 seconds per file and mysql
process is running at 30% CPU every couple of seconds.

As far as the file nodetype is concerned, this is a custom nodetype
which has 4 references per file imported and currently, all the
references are made to the same UUID since we are testing, this could
change in the future.

Any tips or ideas? I will update the results of the test. Right now I
have imported 1K out of 12K files and the import time has gone up to 4
seconds per file. Is this normal? Remember since I am importing the
jackrabbit SVN all files are put under one nt:folder which is
"jackrabbit". This is a pretty normal case of about 12K files and only
78MB. We have plans of a 1TB repository.

Stefan.


On 4/25/07, Stefan Guggisberg <stefan.guggisberg@gmail.com> wrote:
> hi stefan,
>
> On 4/24/07, Stefan Kurla <stefan.kurla@gmail.com> wrote:
> > I am trying to import my jackrabbit svn directory into jackrabbit.
> > This dir has a few extra files like the jackrabbit 1.0 release.
> > Overall we are talking about 713MB on disk with 103K files and 48K
> > folders.
> >
> > I use mysql for persistence and the only thing that gets saved in the
> > filesystem are the indexes. I do a session.save() after importing each
> > file and then I check it in.
> >
> > However when I import all these files into mysql 5.0 (default
> > everything) , I get a failed to write node references: and
> >
> > com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for
> > column 'REFS_DATA' at row 1
> >         at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2868)
>
> you probably hit the size limit of the 'blob' data type. for more information
> see https://issues.apache.org/jira/browse/JCR-760. please note that this
> issue has been fixed in the latest 1.3 release. you can also make those
> changes on an existing database using 'alter table' commands in the
> mysql console.
>
> however, the previous schema allowed for rouglhy 800-1000 references
> (depending on the ref. property name size) on a given target node.
>
> do you explicitly create references? can you share same code (fragements)?
>
> cheers
> stefan
>
>
> >
> > error on the server side.
> >
> > On the client (rmi) side, the error is
> > javax.jcr.RepositoryException: /: unable to update item.: failed to
> > write node references: d5f7e01d-1d68-470e-ba68-02b503754b68
> >         at org.apache.jackrabbit.rmi.server.ServerObject.getRepositoryException(ServerObject.java:136)
> >
> >
> > This happens when the totalFiles imported is 184 and the
> > totalImportSize=437,643 bytes (437KB) and the totalDirs imported are
> > 240.
> >
> > Something is not right.....  jackrabbit cannot croak at 437KB, can it?
> >
> > Please advice.
> >
>

Mime
View raw message