db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Hlavatý <hla...@code.cz>
Subject Re: java1.4.2 "rws" mode - log write performance
Date Sat, 04 Sep 2004 11:17:07 GMT
As this seems crucial for log performance,
I made a test to see for myself what's really the best way to write a log.
It tests various forced sequential write methods, with growing and preallocated file.
Using direct buffers doesn't seem to have any effect over plain RandomAccessFile.write(byte[]).

Following results are from my Athlon XP 1700+ Windows XP Professional machine,
with IDE disk that can dish out unsynced writes at 40MB/s, running JDK1.5 RC.
It proves Mike's point about "rws" being best on preallocated files on Windows.
That is, it can write 10MB/s in 4228 byte chunks, that is about 2490 chunks per sec.
So if you want to get 10000 transactions/sec, you will have to combine logs and
write them in one chunk. I think that's exactly what Derby does. So we're all good ;)
It also shows that sync() / force() are thing to avoid at all costs on Windows ;)

Can anyone run the test on another OS? I've attached the source file for the tests.
(warning: 10000 records test ran for 30minutes ;-) You can decrease the chunk count)


---------------------------------------------
1. System info:
---------------------------------------------

OS Platform: Windows XP/x86/5.1
  Java spec: Java Platform API Specification version 1.5 from Sun Microsystems Inc.
    Java VM: Java HotSpot(TM) Client VM version 1.5.0-rc-b63 from Sun Microsystems Inc.
  Java home: C:\java\jdk1.5.0\jre
  Test file: C:\Documents and Settings\hlavac\iotest\test.bin

Test file exists, deleting...

---------------------------------------------------------------------
2. Testing growing file using RandomAccessFile "rw" + .getFD().sync()
---------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 283527 ms
Writes per second: 35.27 writes/s
   Time per chunk: 28.353 ms
  Write bandwidth: 144.0KB/s

---------------------------------------------------------------------------
3. Testing preallocated file using RandomAccessFile "rw" + .getFD().sync()
---------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 276988 ms
Writes per second: 36.103 writes/s
   Time per chunk: 27.699 ms
  Write bandwidth: 148.0KB/s

---------------------------------------------------------------------
4. Testing growing file using RandomAccessFile "rws"
---------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 9173 ms
Writes per second: 1090.156 writes/s
   Time per chunk: 0.917 ms
  Write bandwidth: 4.0MB/s

---------------------------------------------------------------------
5. Testing preallocated file using RandomAccessFile "rws"
---------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 4016 ms
Writes per second: 2490.04 writes/s
   Time per chunk: 0.402 ms
  Write bandwidth: 10.0MB/s

---------------------------------------------------------------------
6. Testing growing file using RandomAccessFile "rwd"
---------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 10264 ms
Writes per second: 974.279 writes/s
   Time per chunk: 1.026 ms
  Write bandwidth: 3.0MB/s

---------------------------------------------------------------------
7. Testing preallocated file using RandomAccessFile "rwd"
---------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 5598 ms
Writes per second: 1786.352 writes/s
   Time per chunk: 0.56 ms
  Write bandwidth: 7.0MB/s

------------------------------------------------------------------------------
8. Testing growing file using direct buffer + FileChannel "rw" + force(true)
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 281445 ms
Writes per second: 35.531 writes/s
   Time per chunk: 28.144 ms
  Write bandwidth: 144.0KB/s

-----------------------------------------------------------------------------------
9. Testing preallocated file using direct buffer + FileChannel "rw" + force(true)
-----------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 276638 ms
Writes per second: 36.148 writes/s
   Time per chunk: 27.664 ms
  Write bandwidth: 148.0KB/s

------------------------------------------------------------------------------
10. Testing growing file using direct buffer + FileChannel "rw" + force(false)
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 283908 ms
Writes per second: 35.223 writes/s
   Time per chunk: 28.391 ms
  Write bandwidth: 144.0KB/s

------------------------------------------------------------------------------
11. Testing preallocated file using direct buffer + FileChannel "rw" + force(false)
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 277960 ms
Writes per second: 35.976 writes/s
   Time per chunk: 27.796 ms
  Write bandwidth: 144.0KB/s

------------------------------------------------------------------------------
12. Testing growing file using direct buffer + FileChannel "rws"
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 8733 ms
Writes per second: 1145.082 writes/s
   Time per chunk: 0.873 ms
  Write bandwidth: 4.0MB/s

------------------------------------------------------------------------------
13. Testing preallocated file using direct buffer + FileChannel "rws"
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 3805 ms
Writes per second: 2628.121 writes/s
   Time per chunk: 0.381 ms
  Write bandwidth: 10.0MB/s

------------------------------------------------------------------------------
14. Testing growing file using direct buffer + FileChannel "rwd"
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 10595 ms
Writes per second: 943.841 writes/s
   Time per chunk: 1.06 ms
  Write bandwidth: 3.0MB/s

------------------------------------------------------------------------------
15. Testing preallocated file using direct buffer + FileChannel "rwd"
------------------------------------------------------------------------------
       Chunk size: 4228 bytes
            Count: 10000
       Total time: 5508 ms
Writes per second: 1815.541 writes/s
   Time per chunk: 0.551 ms
  Write bandwidth: 7.0MB/s



Mime
View raw message