stdcxx-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Sebor (JIRA)" <j...@apache.org>
Subject [jira] Commented: (STDCXX-142) std::stringsteram insertion of character arrays very slow
Date Thu, 16 Feb 2006 21:02:09 GMT
    [ http://issues.apache.org/jira/browse/STDCXX-142?page=comments#action_12366681 ] 

Martin Sebor commented on STDCXX-142:
-------------------------------------

See also http://gcc.gnu.org/bugzilla/show_bug.cgi?id=26250 for the details of why GNU libstdc++
is so much faster. The gist is that it punts on implementing some rather inconvenient requirements
that the current standard imposes in favor of better performance. Our implementation follows
the standard to the letter and takes a performance hit.

The 2003 standard is being amended/changed so as to allow the more efficient implementation.
See the following defect reports:
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#169
and
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#432

> std::stringsteram insertion of character arrays very slow
> ---------------------------------------------------------
>
>          Key: STDCXX-142
>          URL: http://issues.apache.org/jira/browse/STDCXX-142
>      Project: C++ Standard Library
>         Type: Improvement
>   Components: 27. Input/Output
>     Versions: 4.1.2, 4.1.3
>  Environment: all
>     Reporter: Martin Sebor
>     Assignee: Martin Sebor
>     Priority: Critical
>      Fix For: 4.1.4

>
> From http://mail-archives.apache.org/mod_mbox/incubator-stdcxx-dev/200602.mbox/%3c43ECE5A3.1070609@roguewave.com%3e:
> -------- Original Message --------
> Subject: Benchmarking stdcxx
> Date: Fri, 10 Feb 2006 12:12:35 -0700
> From: Andrew Black <ablack@roguewave.com>
> Reply-To: stdcxx-dev@incubator.apache.org
> To: stdcxx-dev@incubator.apache.org
>   Greetings all.
> I thought it might be interesting to do some benchmarking, comparing the 
> performance of stdcxx with other standard libraries.  As there are a 
> number of attributes that can be compared when doing a benchmark, and an 
> even larger number of classes that can be looked at, there is a fair 
> amount of choice in what to measure.  As a starting point, I chose to 
> measure the runtime performace of stringstream objects.
> Measurements were taken on my linux box (a 1.9 GHz P4), with a light 
> load (number of running applications, but most were idle) and an 8d 
> (single threaded, release, shared) version of stdcxx.  Each test was run 
> 5 times in a row, with a count of 500000 iterations.  The following 
> table lists the run times collected.  All times are in seconds.
> +-------------------+---------------+----------------+
> |     test name     |   gcc 3.2.3   |  stdcxx 4.1.3  |
> +-------------------+-------+-------+--------+-------+
> |                   |   usr | sys   |    usr | sys   |
> +-------------------+-------+-------+--------+-------+
> |    read_single    | 8.977 | 0.008 | 13.997 | 0.012 |
> |                   | 7.856 | 0.008 | 13.913 | 0.016 |
> |                   | 8.021 | 0.012 | 13.817 | 0.024 |
> |                   | 7.736 | 0.020 | 28.634 | 0.016 |
> |                   | 7.844 | 0.012 | 13.841 | 0.016 |
> +-------------------+-------+-------+--------+-------+
> |    read_multi     | 0.608 | 0.744 |  0.864 | 0.756 |
> |                   | 0.688 | 0.704 |  0.860 | 0.736 |
> |                   | 0.660 | 0.728 |  0.856 | 0.712 |
> |                   | 0.608 | 0.792 |  0.848 | 0.724 |
> |                   | 0.552 | 0.796 |  0.796 | 0.780 |
> +-------------------+-------+-------+--------+-------+
> |   write_single    | 1.976 | 0.000 | 30.450 | 0.048 |
> |                   | 2.356 | 0.012 | 30.526 | 0.064 |
> |                   | 1.984 | 0.000 | 30.354 | 0.032 |
> |                   | 1.964 | 0.012 | 30.350 | 0.028 |
> |                   | 1.936 | 0.000 | 30.286 | 0.036 |
> +-------------------+-------+-------+--------+-------+
> |   write_multi     | 1.172 | 2.352 | 32.326 | 2.320 |
> |                   | 1.092 | 2.444 | 31.102 | 2.216 |
> |                   | 1.164 | 2.360 | 30.482 | 2.248 |
> |                   | 1.148 | 2.380 | 31.930 | 2.180 |
> |                   | 1.000 | 2.532 | 29.534 | 2.272 |
> +-------------------+-------+-------+--------+-------+
> | read_write_single | 7.684 | 0.000 | 13.649 | 0.016 |
> |                   | 7.684 | 0.012 | 13.685 | 0.016 |
> |                   | 7.664 | 0.012 | 14.193 | 0.016 |
> |                   | 8.353 | 0.012 | 13.745 | 0.016 |
> |                   | 7.700 | 0.012 | 13.677 | 0.004 |
> +-------------------+-------+-------+--------+-------+
> | read_write_cycle  | 0.056 | 0.000 |  0.412 | 0.000 |
> |                   | 0.056 | 0.000 |  0.424 | 0.004 |
> |                   | 0.056 | 0.000 |  0.428 | 0.004 |
> |                   | 0.056 | 0.000 |  0.420 | 0.004 |
> |                   | 0.056 | 0.000 |  0.412 | 0.004 |
> +-------------------+-------+-------+--------+-------+
> | read_write_multi  | 0.664 | 0.732 |  1.028 | 0.716 |
> |                   | 0.676 | 0.712 |  0.988 | 0.744 |
> |                   | 0.632 | 0.752 |  1.036 | 0.716 |
> |                   | 0.688 | 0.704 |  1.080 | 0.732 |
> |                   | 0.632 | 0.732 |  0.940 | 0.804 |
> +-------------------+-------+-------+--------+-------+
> | write_read_single | 7.868 | 0.016 | 43.407 | 0.044 |
> |                   | 7.896 | 0.012 | 43.895 | 0.044 |
> |                   | 7.888 | 0.008 | 43.307 | 0.076 |
> |                   | 7.912 | 0.012 | 43.391 | 0.032 |
> |                   | 8.337 | 0.016 | 43.375 | 0.044 |
> +-------------------+-------+-------+--------+-------+
> | write_read_cycle  | 0.056 | 0.000 |  0.412 | 0.004 |
> |                   | 0.056 | 0.000 |  0.404 | 0.016 |
> |                   | 0.056 | 0.000 |  0.412 | 0.000 |
> |                   | 0.056 | 0.000 |  0.420 | 0.000 |
> |                   | 0.052 | 0.004 |  0.416 | 0.004 |
> +-------------------+-------+-------+--------+-------+
> | write_read_multi  | 7.340 | 2.404 | 43.591 | 2.408 |
> |                   | 7.420 | 2.400 | 42.347 | 2.196 |
> |                   | 7.440 | 2.376 | 45.227 | 2.336 |
> |                   | 7.232 | 2.476 | 43.679 | 2.316 |
> |                   | 7.348 | 2.488 | 44.271 | 2.348 |
> +-------------------+-------+-------+--------+-------+
> Analysis:
> Using the numbers above, I did some basic analysis.  System times spent 
> for a given test appear to be roughly the same, so I am overlooking 
> those numbers at this time.
> To look at these numbers, I see two or three stastical operations that 
> could be of use.
> The first operation is the arithmatic average ('average') of the 
> numbers.  This is the 'classic'  sum and divide average.  The second 
> operation is the medan value (middle number) in the set.  The final 
> operation is what I term the 'middle average'.  I calculate this by 
> throwing out the highest and lowest value, then calculating the 
> arithmatic average of the remaining numbers.
> In the tables below, ratio indicates how much longer the stdcxx runs 
> take compared to the gcc runs, with 0% indicating they take the same 
> amount of time.
> +-------------------+-------+--------+----------+
> |    read_single    |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 8.087 | 16.840 |  108.25% |
> +-------------------+-------+--------+----------+
> |   middle average  | 7.907 | 13.917 |   76.01% |
> +-------------------+-------+--------+----------+
> |       medan       | 7.856 | 13.913 |   77.10% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> |    read_multi     |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 0.623 |  0.845 |   35.56% |
> +-------------------+-------+--------+----------+
> |   middle average  | 0.625 |  0.855 |   36.67% |
> +-------------------+-------+--------+----------+
> |       medan       | 0.608 |  0.856 |   40.79% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> |   write_single    |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 2.043 | 30.393 | 1387.53% |
> +-------------------+-------+--------+----------+
> |   middle average  | 1.975 | 30.385 | 1438.72% |
> +-------------------+-------+--------+----------+
> |       medan       | 1.976 | 30.354 | 1436.13% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> |   write_multi     |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 1.115 | 31.075 | 2686.48% |
> +-------------------+-------+--------+----------+
> |   middle average  | 1.135 | 31.171 | 2647.18% |
> +-------------------+-------+--------+----------+
> |       medan       | 1.148 | 31.102 | 2609.23% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | read_write_single |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 7.817 | 13.790 |   76.41% |
> +-------------------+-------+--------+----------+
> |   middle average  | 7.689 | 13.720 |   78.20% |
> +-------------------+-------+--------+----------+
> |       medan       | 7.684 | 13.685 |   78.10% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | read_write_cycle  |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 0.056 |  0.419 |  648.57% |
> +-------------------+-------+--------+----------+
> |   middle average  | 0.056 |  0.419 |  647.62% |
> +-------------------+-------+--------+----------+
> |       medan       | 0.056 |  0.420 |  650.00% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | read_write_multi  |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 0.658 |  1.014 |   54.07% |
> +-------------------+-------+--------+----------+
> |   middle average  | 0.657 |  1.017 |   54.77% |
> +-------------------+-------+--------+----------+
> |       medan       | 0.664 |  1.028 |   54.82% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | write_read_single |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 7.980 | 43.475 |  444.79% |
> +-------------------+-------+--------+----------+
> |   middle average  | 7.899 | 43.391 |  449.35% |
> +-------------------+-------+--------+----------+
> |       medan       | 7.896 | 43.391 |  449.53% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | write_read_cycle  |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 0.055 |  0.413 |  647.83% |
> +-------------------+-------+--------+----------+
> |   middle average  | 0.056 |  0.413 |  638.10% |
> +-------------------+-------+--------+----------+
> |       medan       | 0.056 |  0.412 |  635.71% |
> +-------------------+-------+--------+----------+
> +-------------------+-------+--------+----------+
> | write_read_multi  |  gcc  | stdcxx |   ratio  |
> +-------------------+-------+--------+----------+
> |      average      | 7.356 | 43.823 |  495.74% |
> +-------------------+-------+--------+----------+
> |   middle average  | 7.369 | 43.847 |  494.99% |
> +-------------------+-------+--------+----------+
> |       medan       | 7.348 | 43.679 |  494.43% |
> +-------------------+-------+--------+----------+
> Conclusions:
> Looking over the processed numbers from the runs, one thing that jumps 
> out at me is the write times, particularly the write_single and 
> write_multi benchmarks.  Both of these benchmarks are an order of 
> magnitude slower than their GCC counterparts (at least on this 
> computer).  The write_multi benchmark in particular shows what happens 
> if you stream large amounts of data (~250 MB worth of data in this case) 
> into a strstream, without streaming any out.
> Future:
> For those interested in trying to repeat these tests, I have attached 
> the source and makefile files I used to generate these benchmarks.  This 
> particular benchmark is a work in progress.  There are several 
> additional things that could be benchmarked regarding stringstreams.  
> These include allocation (default, string, copy), pseudo-random 
> read/writes (rather than pattern read/writes), reads and writes of 
> varying length strings, and reading/writing using something other than 
> the insertion and extraction operators.
> --Andrew Black

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message