jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jukka Zitting <jukka.zitt...@gmail.com>
Subject Re: Some more benchmarks
Date Mon, 03 Jun 2013 08:27:23 GMT
Hi,

On Mon, Jun 3, 2013 at 11:09 AM, Thomas Mueller <mueller@adobe.com> wrote:
> A bit weird is, when I run the tests separately I get different numbers:

The results depend on the hardware you're using, so in general numbers
from two different environments are not directly comparable.

> In your case, the N was 304 versus 3574 (more than 10 times different), in
> my case it was 528 versus 1085 (factor 2).

Even relative numbers across fixtures can be different depending on
the varying IO/CPU/memory access costs on different environments. For
example an SSD disk will reduce the relative advantage of the TarMK
that "cheats" by mmapping the entire repository to memory.

> How did you run the test? I will try the same command line and post my
> results.

I'm using a ec2 m1.medium instance to keep the environment stable over
time. It would be nice to keep track of results also on different
hardware.

The command line I've used so far is simply:

    $ java -jar oak-run-*.jar benchmark \
          ReadPropertyTest SmallFileReadTest SmallFileWriteTest \
          Jackrabbit Oak-Default Oak-Mongo Oak-Segment Oak-Tar

BR,

Jukka Zitting

Mime
View raw message