zipkin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-zipkin] anuraaga edited a comment on issue #2579: Benchmark bytes / bytebuffer, protobuf vs zipkin vs wire.
Date Fri, 10 May 2019 11:17:14 GMT
anuraaga edited a comment on issue #2579: Benchmark bytes / bytebuffer, protobuf vs zipkin
vs wire.
URL: https://github.com/apache/incubator-zipkin/pull/2579#issuecomment-491252980
 
 
   Yeah it'd definitely only be for the server. And while `protobuf-java-util` has some dependencies,
just `protobuf-java` is dependency free and I believe has never broken backwards compatibility
so I wouldn't write it off so fast ;)
   
   I added a wire-based decoder too, with the same code flow as the protobuf-based one, just
uses wire for the decoding. Now that it is doing hex decoding, it's a proper comparison. Wire
is quite a bit slower, probably since it has to do an array copy before converting to hex
since it doesn't expose the ability to read single bytes, and maybe because protobuf uses
`Unsafe` though I don't know if that really makes a big impact on modern JVMs (it would have
a larger impact with java 8 than java 11 I guess).
   
   Protobuf vs zipkin seems to show protobuf a few percent faster. 
   
   Garbage-wise, as expected the lack of an initial array copy does remove a whole buffer's
worth of allocation, but due to the lots of allocations to create hex and other strings, it's
a very small percentage overall, ~3%, so just a little improvement. This is after applying
a change to reuse `char[]`, which had a similar improvement in the base case too.
   
   There do seem to be some differences shown here, though they're not so huge as to make
it clear whether any approach is the best.
   
   ```
   Benchmark                                                                           (num)
   Mode   Cnt         Score         Error   Units
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder                                     10000
 sample  1309     11510.855 ±     138.341   us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.00    10000
 sample           10010.624                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.50    10000
 sample           10977.280                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.90    10000
 sample           13156.352                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.95    10000
 sample           15319.040                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.99    10000
 sample           16967.270                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.999   10000
 sample           24244.060                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p0.9999  10000
 sample           25493.504                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:bytes_copy_protobufDecoder·p1.00    10000
 sample           25493.504                 us/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.alloc.rate                      10000
 sample    15      1212.778 ±      22.951  MB/sec
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.alloc.rate.norm                 10000
 sample    15  21949689.300 ±     213.505    B/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Eden_Space             10000
 sample    15      1110.207 ±     113.046  MB/sec
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Eden_Space.norm        10000
 sample    15  20094766.898 ± 2030230.821    B/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Old_Gen                10000
 sample    15       112.514 ±      11.349  MB/sec
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Old_Gen.norm           10000
 sample    15   2036516.677 ±  203864.604    B/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Survivor_Space         10000
 sample    15         0.443 ±       0.438  MB/sec
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.churn.G1_Survivor_Space.norm    10000
 sample    15      8009.395 ±    7904.907    B/op
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.count                           10000
 sample    15        82.000                counts
   ProtoCodecBenchmarks.bytes_copy_protobufDecoder:·gc.time                            10000
 sample    15       346.000                    ms
   ProtoCodecBenchmarks.bytes_copy_wireDecoder                                         10000
 sample  1068     14170.764 ±     135.522   us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.00            10000
 sample           12648.448                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.50            10000
 sample           13713.408                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.90            10000
 sample           15810.560                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.95            10000
 sample           16846.029                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.99            10000
 sample           19930.808                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.999           10000
 sample           24186.126                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p0.9999          10000
 sample           24281.088                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:bytes_copy_wireDecoder·p1.00            10000
 sample           24281.088                 us/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.alloc.rate                          10000
 sample    15      1129.202 ±      28.012  MB/sec
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.alloc.rate.norm                     10000
 sample    15  25120909.386 ±     202.273    B/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Eden_Space                 10000
 sample    15      1040.511 ±      54.956  MB/sec
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Eden_Space.norm            10000
 sample    15  23153464.194 ± 1194946.337    B/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Old_Gen                    10000
 sample    15        90.519 ±       4.650  MB/sec
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Old_Gen.norm               10000
 sample    15   2014291.467 ±  102517.266    B/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Survivor_Space             10000
 sample    15         0.838 ±       0.323  MB/sec
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.churn.G1_Survivor_Space.norm        10000
 sample    15     18578.527 ±    6846.810    B/op
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.count                               10000
 sample    15        76.000                counts
   ProtoCodecBenchmarks.bytes_copy_wireDecoder:·gc.time                                10000
 sample    15       173.000                    ms
   ProtoCodecBenchmarks.bytes_protobufDecoder                                          10000
 sample  1340     11252.910 ±     126.595   us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.00              10000
 sample            9912.320                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.50              10000
 sample           10731.520                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.90              10000
 sample           12546.867                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.95              10000
 sample           14974.976                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.99              10000
 sample           16912.548                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.999             10000
 sample           19200.541                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p0.9999            10000
 sample           19267.584                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:bytes_protobufDecoder·p1.00              10000
 sample           19267.584                 us/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.alloc.rate                           10000
 sample    15      1130.991 ±      25.879  MB/sec
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.alloc.rate.norm                      10000
 sample    15  20009660.550 ±     259.757    B/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Eden_Space                  10000
 sample    15      1136.615 ±     111.000  MB/sec
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Eden_Space.norm             10000
 sample    15  20105313.287 ± 1867925.592    B/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Old_Gen                     10000
 sample    15         0.319 ±       0.197  MB/sec
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Old_Gen.norm                10000
 sample    15      5659.194 ±    3507.786    B/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Survivor_Space              10000
 sample    15         0.354 ±       0.454  MB/sec
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.churn.G1_Survivor_Space.norm         10000
 sample    15      6261.311 ±    8110.558    B/op
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.count                                10000
 sample    15        84.000                counts
   ProtoCodecBenchmarks.bytes_protobufDecoder:·gc.time                                 10000
 sample    15       352.000                    ms
   ProtoCodecBenchmarks.bytes_wireDecoder                                              10000
 sample  1027     14696.033 ±     274.890   us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.00                      10000
 sample           12697.600                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.50                      10000
 sample           13991.936                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.90                      10000
 sample           16420.045                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.95                      10000
 sample           17812.685                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.99                      10000
 sample           26746.552                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.999                     10000
 sample           63427.052                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p0.9999                    10000
 sample           64225.280                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:bytes_wireDecoder·p1.00                      10000
 sample           64225.280                 us/op
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.alloc.rate                               10000
 sample    15      1003.950 ±      33.760  MB/sec
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.alloc.rate.norm                          10000
 sample    15  23180993.238 ±     327.812    B/op
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Eden_Space                      10000
 sample    15      1001.456 ±     113.305  MB/sec
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Eden_Space.norm                 10000
 sample    15  23105838.211 ± 2265066.780    B/op
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Old_Gen                         10000
 sample    15         0.585 ±       0.150  MB/sec
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Old_Gen.norm                    10000
 sample    15     13499.422 ±    3403.060    B/op
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Survivor_Space                  10000
 sample    15         0.752 ±       0.249  MB/sec
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.churn.G1_Survivor_Space.norm             10000
 sample    15     17272.859 ±    5177.973    B/op
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.count                                    10000
 sample    15        73.000                counts
   ProtoCodecBenchmarks.bytes_wireDecoder:·gc.time                                     10000
 sample    15       158.000                    ms
   ProtoCodecBenchmarks.bytes_zipkinDecoder                                            10000
 sample  1220     12361.137 ±     243.588   us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.00                  10000
 sample            9863.168                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.50                  10000
 sample           11452.416                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.90                  10000
 sample           15382.938                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.95                  10000
 sample           18246.861                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.99                  10000
 sample           22308.127                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.999                 10000
 sample           26914.292                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p0.9999                10000
 sample           27066.368                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:bytes_zipkinDecoder·p1.00                  10000
 sample           27066.368                 us/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.alloc.rate                             10000
 sample    15       984.706 ±      72.183  MB/sec
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.alloc.rate.norm                        10000
 sample    15  19129801.132 ±     226.638    B/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Eden_Space                    10000
 sample    15       993.104 ±     126.048  MB/sec
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Eden_Space.norm               10000
 sample    15  19270497.313 ± 1683336.845    B/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Old_Gen                       10000
 sample    15         0.434 ±       0.268  MB/sec
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Old_Gen.norm                  10000
 sample    15      8434.596 ±    5290.023    B/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Survivor_Space                10000
 sample    15         0.753 ±       0.651  MB/sec
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.churn.G1_Survivor_Space.norm           10000
 sample    15     14673.383 ±   12535.509    B/op
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.count                                  10000
 sample    15        83.000                counts
   ProtoCodecBenchmarks.bytes_zipkinDecoder:·gc.time                                   10000
 sample    15       335.000                    ms

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message