jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nuescheler <david.nuesche...@gmail.com>
Subject Re: the 1mn test: first jackrabbit performance and scalability tests
Date Fri, 12 Nov 2004 09:22:04 GMT
> >> I ran the code you posted on my local box and the results
> >> are mostly comparable. However, I got a big performance
> >> boost when using the LocalFileSystem on a Reiser4
> >> filesystem (Linux 2.6.8): between 4 and 5 times faster
> >> then ext3!
> > cool. very interesting.
> I performed the load test comparing the
> ObjectPersistenceManager+CQFileSystem vs.
> ObjectPersistenceManager+LocalFileSystem on Reiser4. The surprising
> result is that they go at about the same speed! LFS on Reiser4 starts
> at about 5s/1000n and grows slowly from that. After about 50k nodes,
> the speed is around 9s. CQFS is comparable. I suspect the slowdown is
> due to the memory consumption of the JVM growing rather than to the
> number of files on disk.
> Now, I find this very interesting, since I'd rather use a /real/
> filesystem than something like CQFileSystem, which puts everything
> inside one big file: I suspect this to be unmanageable when the size of
> your repository becomes quite large. Please correct me if I have the
> wrong impression.
well, the thing is that the cqfs is a pure java filesystem which is built
pretty much exactly the same way as your other unix-filesystems
(inodes, blocks, indirect blocks, ...), so the scalability patterns are
very similar to your average unix fs, but cqfs can work with a much 
smaller blocksize than 512 bytes.
now, while i agree it would be the best solution to have a kernel 
supported fast native fs, unfortunately this is not always 
possible (eg. windows). so let the cqfs be the "poor mans", platform 
independent "reiser substitute" ;)

> I also suspect that it would be hard to beat these times with a JDBC
> persistence manager.
i most certainly agree these performance metrics will be hard to
beat for anything that is connected through a network layer to the 
repository, since i believe that already the tcp overhead is beyond
the benchmark times that we are looking at now.

> Finally, maybe the LocalFileSystem class can be optimized further, what
> do you think?
most certainly, i believe that there is still a lot of
space for performance improvement. 

however:
a) repositories in most applications that i can think of
are used in a mostly "reading" environment. so while
this 1mn benchmark is interesting it does not reflect 
true common usage patterns in a repository.

b) there are still a couple of functional blocks that
are unfinished and we are still manipulating the core
quite a bit so i am not sure whether it is the time to
do performance tuning already, especially since i
think we are in a somewhat acceptable range already.

c) a fast persistence manager takes up only a
small pecentage (less than 20%) of the overall
consumed time. the rest is spent in other
parts of jackrabbit so we might also want to
look at performance optimizations in other
parts.

regards,
david

----------------------------------------------------------------------
standardize your content-repository !
                               http://www.jcp.org/en/jsr/detail?id=170
---------------------------------------< david.nuescheler@day.com >---

This message is a private communication. If you are not the intended
recipient, please do not read, copy, or use it, and do not disclose it
to others. Please notify the sender of the delivery error by replying
to this message, and then delete it from your system. Thank you.

The sender does not assume any liability for timely, trouble free,
complete, virus free, secure, error free or uninterrupted arrival of
this e-mail. For verification please request a hard copy version.


mailto:david.nuescheler@day.com
http://www.day.com

David Nuescheler
Chief Technology Officer
Day Software AG
Barfuesserplatz 6 / Postfach
4001 Basel
Switzerland

T  41 61 226 98 98
F  41 61 226 98 97

Mime
View raw message