stdcxx-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Chariton <chari...@roguewave.com>
Subject Benchmarking stdcxx for Windows
Date Wed, 05 Apr 2006 22:16:10 GMT
I took Andrew Black's posting of February 10, 2006 concerning benchmarking
stdcxx and applied it to Windows.  I decided to benchmark stdcxx 4.1.3 prior
to the modifications made following Andrew's initial tests.  I will then
modify the library with the updates designed to resolve some of the
performance issues and run these tests again to determine what effect, if
any, the modifications have on performance. 

 

I modified Andrew's stringstream_bm.cpp file to work on Windows.  I then ran
it using the MSVC7.1 compiler with the stdcxx library and the native
library.  The tests were conducted on my Dell Optiplex GX280 running Windows
XP. I ran each test five times in a row with a count of 500000 iterations,
just as Andrew did.

 

The following table lists the run times collected.  All times are in
seconds.

 

+-------------------+---------------+----------------+

|     test name     |   MSVC 7.1    |  stdcxx 4.1.3  |

+-------------------+-------+-------+--------+-------+

|                   |   usr | sys   |    usr | sys   |

+-------------------+-------+-------+--------+-------+

|    read_single    | 4.156 | 0.156 | 12.500 | 0.000 |

|                   | 4.156 | 0.000 | 12.500 | 0.000 |

|                   | 4.156 | 0.156 | 12.500 | 0.000 |

|                   | 4.156 | 0.000 | 12.484 | 0.000 |

|                   | 4.156 | 0.000 | 12.484 | 0.000 |

+-------------------+-------+-------+--------+-------+

|    read_multi     | 0.328 | 0.109 |  0.359 | 0.109 |

|                   | 0.281 | 0.141 |  0.406 | 0.781 |

|                   | 0.375 | 0.625 |  0.406 | 0.625 |

|                   | 0.344 | 0.938 |  0.344 | 0.141 |

|                   | 0.281 | 0.141 |  0.375 | 0.938 |

+-------------------+-------+-------+--------+-------+

|   write_single    | 2.391 | 0.000 | 13.500 | 0.000 |

|                   | 2.391 | 0.000 | 13.500 | 0.000 |

|                   | 2.391 | 0.000 | 13.500 | 0.000 |

|                   | 2.375 | 0.000 | 13.484 | 0.000 |

|                   | 2.375 | 0.000 | 13.500 | 0.000 |

+-------------------+-------+-------+--------+-------+

|   write_multi     | 0.859 | 0.641 | 12.578 | 0.484 |

|                   | 0.875 | 0.594 | 12.578 | 0.484 |

|                   | 0.875 | 0.563 | 12.547 | 0.516 |

|                   | 0.844 | 0.609 | 12.563 | 0.484 |

|                   | 0.891 | 0.578 | 12.594 | 0.484 |

+-------------------+-------+-------+--------+-------+

| read_write_single | 4.297 | 0.000 | 12.578 | 0.000 |

|                   | 4.297 | 0.000 | 13.685 | 0.000 |

|                   | 4.297 | 0.000 | 12.609 | 0.000 |

|                   | 4.297 | 0.000 | 12.625 | 0.000 |

|                   | 4.313 | 0.000 | 12.609 | 0.156 |

+-------------------+-------+-------+--------+-------+

| read_write_cycle  | 0.281 | 0.000 |  0.313 | 0.000 |

|                   | 0.297 | 0.000 |  0.313 | 0.157 |

|                   | 0.281 | 0.156 |  0.313 | 0.000 |

|                   | 0.281 | 0.000 |  0.313 | 0.000 |

|                   | 0.281 | 0.000 |  0.313 | 0.000 |

+-------------------+-------+-------+--------+-------+

| read_write_multi  | 0.578 | 0.109 |  0.484 | 0.125 |

|                   | 0.609 | 0.625 |  0.453 | 0.156 |

|                   | 0.656 | 0.313 |  0.516 | 0.938 |

|                   | 0.641 | 0.469 |  0.547 | 0.781 |

|                   | 0.578 | 0.938 |  0.500 | 0.109 |

+-------------------+-------+-------+--------+-------+

| write_read_single | 5.703 | 0.000 | 25.203 | 0.000|

|                   | 5.688 | 0.000 | 25.172 | 0.000 |

|                   | 5.703 | 0.000 | 25.188 | 0.000 |

|                   | 5.703 | 0.000 | 25.188 | 0.000 |

|                   | 5.703 | 0.000 | 25.188 | 0.000 |

+-------------------+-------+-------+--------+-------+

| write_read_cycle  | 0.281 | 0.000 |  0.313 | 0.000 |

|                   | 0.281 | 0.000 |  0.328 | 0.000 |

|                   | 0.281 | 0.000 |  0.313 | 0.000 |

|                   | 0.281 | 0.000 |  0.313 | 0.156 |

|                   | 0.281 | 0.000 |  0.297 | 0.156 |

+-------------------+-------+-------+--------+-------+

| write_read_multi  | 4.203 | 0.531 | 24.188 | 0.453 |

|                   | 4.781 | 0.656 | 24.188 | 0.453 |

|                   | 4.313 | 0.703 | 24.938 | 0.531 |

|                   | 4.141 | 0.594 | 24.172 | 0.453 |

|                   | 4.188 | 0.563 | 24.125 | 0.516 |

+-------------------+-------+-------+--------+-------+

 

Analysis:

Using the numbers above, I did some basic analysis along the lines that
Andrew Black did.  System times spent for a given test appear to be roughly
the same, so I am overlooking those numbers at this time, the same as Andrew
did.

 

I calculated the arithmetic average and median values for these numbers.  I
also calculated the ratio using the same method that Andrew did.  The ratio
indicates how much longer the stdcxx runs take compared to the MSVC 7.1
runs, with 0% indicating that they take the same amount of time.

 

+-------------------+-------+--------+----------+

|    read_single    |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 4.156 | 12.494 |  200.60% |

+-------------------+-------+--------+----------+

|       median      | 4.156 | 12.500 |  200.75% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

|    read_multi     |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 0.328 |  0.378 |   15.24% |

+-------------------+-------+--------+----------+

|       median      | 0.344 |  0.375 |    9.09% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

|   write_single    |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 2.384 | 13.497 |  466.06% |

+-------------------+-------+--------+----------+

|       median      | 2.391 | 13.500 |  464.71% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

|   write_multi     |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 0.869 | 12.572 | 1347.12% |

+-------------------+-------+--------+----------+

|       median      | 0.875 | 12.578 | 1337.50% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| read_write_single |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 4.300 | 12.610 |  193.31% |

+-------------------+-------+--------+----------+

|       median      | 4.297 | 12.609 |  193.45% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| read_write_cycle  |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 0.284 |  0.313 |    9.89% |

+-------------------+-------+--------+----------+

|       median      | 0.281 |  0.313 |   11.11% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| read_write_multi  |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 0.613 |  0.500 |  -18.37% |

+-------------------+-------+--------+----------+

|       median      | 0.609 |  0.500 |  -17.95% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| write_read_single |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 5.700 | 25.188 |  341.89% |

+-------------------+-------+--------+----------+

|       median      | 5.703 | 25.1881|  341.64% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| write_read_cycle  |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 0.281 |  0.313 |   11.11% |

+-------------------+-------+--------+----------+

|       median      | 0.281 |  0.313 |   11.11% |

+-------------------+-------+--------+----------+

 

+-------------------+-------+--------+----------+

| write_read_multi  |  MSVC | stdcxx |   ratio  |

+-------------------+-------+--------+----------+

|      average      | 4.325 | 24.322 |  462.36% |

+-------------------+-------+--------+----------+

|       median      | 4.203 | 24.188 |  475.46% |

+-------------------+-------+--------+----------+

 

I am going to incorporate the changes suggested by Martin Sebor and rerun
the tests to see how performance improves.

 

Craig Chariton

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message