Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Profiler] Implement interning API #917

Merged
merged 29 commits into from
Mar 25, 2025
Merged

[Profiler] Implement interning API #917

merged 29 commits into from
Mar 25, 2025

Conversation

danielsn
Copy link
Contributor

What does this PR do?

Adds a new API for adding values to the profiler which interns the value, and then returns a handle to the interned value.

Motivation

The current API has two notable downsides

  1. It requires the creation of a complex nested data-structure
  2. It does not offer any way to take advantage of the interning capabilities of the profiler. Even if we know that two labels / strings / stack frames are the same, we re-add them every time.

This new API follows more of a builder pattern: you give a value, intern it, and then get a handle to the interned value which you can then reuse as needed.

Additional Notes

The new API is twice as fast as the old one on the C/C++ example, which do similar work

./profiles  10.85s user 0.35s system 98% cpu 11.366 total
./profile_intern  4.82s user 0.26s system 96% cpu 5.244 total

How to test the change?

The new C++ file that drives the API.

@danielsn danielsn requested review from a team as code owners March 11, 2025 13:47
@github-actions github-actions bot added the profiling Relates to the profiling* modules. label Mar 11, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 11, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-03-25 13:16:58

Comparing candidate commit 73ac3d5 in PR branch dsn/r_and_d_week_mar_2024 with baseline commit c0f1e51 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 54.159ms 54.435ms ± 0.114ms 54.436ms ± 0.099ms 54.536ms 54.587ms 54.678ms 54.741ms 0.56% -0.022 -0.869 0.21% 0.008ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [54.419ms; 54.451ms] or [-0.029%; +0.029%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 65.326µs 65.543µs ± 0.194µs 65.527µs ± 0.050µs 65.575µs 65.671µs 65.960µs 67.832µs 3.52% 8.760 97.221 0.30% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [65.516µs; 65.570µs] or [-0.041%; +0.041%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 244.568ns 252.630ns ± 10.118ns 248.848ns ± 2.601ns 253.862ns 279.207ns 290.197ns 291.083ns 16.97% 2.424 5.413 3.99% 0.715ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [251.228ns; 254.032ns] or [-0.555%; +0.555%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.190µs 3.172µs ± 1.421µs 2.979µs ± 0.024µs 3.002µs 3.651µs 13.881µs 14.905µs 400.30% 7.408 55.818 44.69% 0.100µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.975µs; 3.369µs] or [-6.209%; +6.209%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.855µs 25.902µs ± 10.291µs 18.251µs ± 0.282µs 35.448µs 44.130µs 46.005µs 71.043µs 289.26% 1.110 1.037 39.63% 0.728µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.476µs; 27.329µs] or [-5.506%; +5.506%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.362µs 2.422µs ± 0.013µs 2.423µs ± 0.004µs 2.427µs 2.441µs 2.445µs 2.446µs 0.96% -1.935 6.616 0.53% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.420µs; 2.424µs] or [-0.074%; +0.074%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 147.262µs 148.044µs ± 0.284µs 147.998µs ± 0.116µs 148.129µs 148.518µs 148.765µs 150.186µs 1.48% 2.411 16.030 0.19% 0.020µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [148.005µs; 148.083µs] or [-0.027%; +0.027%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.910µs 4.991µs ± 0.044µs 4.989µs ± 0.044µs 5.037µs 5.047µs 5.049µs 5.080µs 1.83% -0.007 -1.480 0.87% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [4.985µs; 4.997µs] or [-0.121%; +0.121%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.895µs 3.915µs ± 0.002µs 3.915µs ± 0.001µs 3.916µs 3.918µs 3.920µs 3.921µs 0.15% -2.799 22.991 0.06% 0.000µs 1 200
credit_card/is_card_number/ throughput 255067583.925op/s 255451620.659op/s ± 153666.129op/s 255447211.859op/s ± 86195.675op/s 255533609.058op/s 255639710.943op/s 255791555.622op/s 256738325.968op/s 0.51% 2.829 23.302 0.06% 10865.836op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 81.984µs 82.636µs ± 0.348µs 82.599µs ± 0.256µs 82.858µs 83.230µs 83.468µs 83.993µs 1.69% 0.622 0.386 0.42% 0.025µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11905820.277op/s 12101460.472op/s ± 50760.606op/s 12106740.602op/s ± 37596.529op/s 12141838.608op/s 12173327.874op/s 12188164.543op/s 12197509.269op/s 0.75% -0.597 0.318 0.42% 3589.317op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 76.530µs 77.106µs ± 0.360µs 77.056µs ± 0.242µs 77.336µs 77.775µs 78.009µs 78.119µs 1.38% 0.605 -0.237 0.47% 0.025µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 12800941.519op/s 12969358.816op/s ± 60468.371op/s 12977519.717op/s ± 40660.645op/s 13016671.433op/s 13050077.062op/s 13063514.365op/s 13066746.247op/s 0.69% -0.586 -0.271 0.47% 4275.760op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.896µs 3.913µs ± 0.003µs 3.914µs ± 0.001µs 3.915µs 3.917µs 3.918µs 3.920µs 0.15% -1.829 8.871 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255132405.231op/s 255528717.376op/s ± 168998.625op/s 255508597.865op/s ± 75174.286op/s 255577376.141op/s 255839096.942op/s 255931223.394op/s 256650899.629op/s 0.45% 1.844 8.985 0.07% 11950.007op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 72.929µs 73.710µs ± 0.423µs 73.678µs ± 0.250µs 73.929µs 74.564µs 74.789µs 74.842µs 1.58% 0.520 -0.036 0.57% 0.030µs 1 200
credit_card/is_card_number/378282246310005 throughput 13361402.759op/s 13567101.272op/s ± 77586.206op/s 13572496.705op/s ± 45992.437op/s 13618462.891op/s 13683913.550op/s 13709274.035op/s 13711898.692op/s 1.03% -0.491 -0.070 0.57% 5486.173op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.887µs 52.149µs ± 0.090µs 52.147µs ± 0.064µs 52.212µs 52.298µs 52.349µs 52.352µs 0.39% -0.044 -0.182 0.17% 0.006µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19101463.192op/s 19175806.232op/s ± 33105.187op/s 19176523.145op/s ± 23437.768op/s 19198302.871op/s 19228601.969op/s 19257860.813op/s 19272783.668op/s 0.50% 0.053 -0.175 0.17% 2340.890op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.429µs 6.555µs ± 0.077µs 6.556µs ± 0.050µs 6.597µs 6.695µs 6.735µs 6.766µs 3.20% 0.369 -0.339 1.16% 0.005µs 1 200
credit_card/is_card_number/x371413321323331 throughput 147804041.751op/s 152567237.954op/s ± 1773633.901op/s 152536931.201op/s ± 1150028.557op/s 154036420.754op/s 155298567.642op/s 155424225.674op/s 155549001.104op/s 1.97% -0.317 -0.410 1.16% 125414.856op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.899µs 3.914µs ± 0.002µs 3.914µs ± 0.001µs 3.915µs 3.917µs 3.919µs 3.919µs 0.14% -1.271 6.979 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255163160.669op/s 255513119.197op/s ± 156724.709op/s 255512627.457op/s ± 85276.640op/s 255597380.127op/s 255714997.603op/s 255928791.135op/s 256491435.986op/s 0.38% 1.285 7.065 0.06% 11082.110op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 63.506µs 63.941µs ± 0.352µs 63.898µs ± 0.124µs 64.046µs 64.295µs 64.509µs 68.085µs 6.55% 8.305 94.688 0.55% 0.025µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 14687458.370op/s 15639929.676op/s ± 82378.954op/s 15650027.116op/s ± 30399.064op/s 15677344.669op/s 15710675.250op/s 15738989.911op/s 15746568.903op/s 0.62% -7.857 87.623 0.53% 5825.072op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.402µs 57.555µs ± 0.100µs 57.541µs ± 0.046µs 57.589µs 57.691µs 57.784µs 58.479µs 1.63% 4.248 34.843 0.17% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17100221.153op/s 17374683.300op/s ± 29979.402op/s 17378793.527op/s ± 13921.535op/s 17392147.884op/s 17405208.596op/s 17415918.682op/s 17420987.063op/s 0.24% -4.152 33.644 0.17% 2119.864op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.915µs ± 0.003µs 3.915µs ± 0.001µs 3.916µs 3.918µs 3.920µs 3.934µs 0.49% -0.161 23.954 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254201108.595op/s 255444822.202op/s ± 179029.899op/s 255451489.764op/s ± 71481.752op/s 255515324.532op/s 255627241.019op/s 255796807.945op/s 256751303.151op/s 0.51% 0.216 23.996 0.07% 12659.326op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.576µs 54.771µs ± 0.197µs 54.702µs ± 0.051µs 54.761µs 55.156µs 55.528µs 55.640µs 1.72% 2.054 3.983 0.36% 0.014µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17972542.081op/s 18258067.147op/s ± 65167.543op/s 18281032.516op/s ± 16931.836op/s 18294732.492op/s 18312031.646op/s 18316571.057op/s 18323187.994op/s 0.23% -2.035 3.873 0.36% 4608.041op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.827µs 52.164µs ± 0.108µs 52.181µs ± 0.065µs 52.239µs 52.311µs 52.358µs 52.387µs 0.39% -0.644 0.291 0.21% 0.008µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19088794.410op/s 19170271.019op/s ± 39677.777op/s 19163925.828op/s ± 23927.316op/s 19191683.766op/s 19241626.545op/s 19280319.116op/s 19294847.683op/s 0.68% 0.655 0.311 0.21% 2805.642op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.430µs 6.554µs ± 0.079µs 6.543µs ± 0.062µs 6.607µs 6.699µs 6.747µs 6.761µs 3.33% 0.463 -0.566 1.20% 0.006µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 147904131.231op/s 152601112.611op/s ± 1823374.177op/s 152830619.727op/s ± 1442117.161op/s 154149229.148op/s 155265125.944op/s 155386073.172op/s 155515100.229op/s 1.76% -0.420 -0.627 1.19% 128932.025op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.914µs; 3.915µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/ throughput [255430324.011op/s; 255472917.306op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [82.588µs; 82.684µs] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12094425.541op/s; 12108495.404op/s] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [77.057µs; 77.156µs] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/ 378282246310005 throughput [12960978.482op/s; 12977739.151op/s] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/37828224631 execution_time [3.913µs; 3.914µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631 throughput [255505295.792op/s; 255552138.961op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/378282246310005 execution_time [73.652µs; 73.769µs] or [-0.079%; +0.079%] None None None
credit_card/is_card_number/378282246310005 throughput [13556348.570op/s; 13577853.974op/s] or [-0.079%; +0.079%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.137µs; 52.162µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19171218.171op/s; 19180394.292op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.545µs; 6.566µs] or [-0.162%; +0.162%] None None None
credit_card/is_card_number/x371413321323331 throughput [152321429.354op/s; 152813046.555op/s] or [-0.161%; +0.161%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.913µs; 3.914µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ throughput [255491398.660op/s; 255534839.735op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [63.892µs; 63.989µs] or [-0.076%; +0.076%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15628512.745op/s; 15651346.606op/s] or [-0.073%; +0.073%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.541µs; 57.569µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17370528.443op/s; 17378838.157op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255420010.380op/s; 255469634.024op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.744µs; 54.798µs] or [-0.050%; +0.050%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18249035.552op/s; 18267098.741op/s] or [-0.049%; +0.049%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.149µs; 52.179µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19164772.061op/s; 19175769.977op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.543µs; 6.565µs] or [-0.166%; +0.166%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [152348410.486op/s; 152853814.735op/s] or [-0.166%; +0.166%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.273µs 505.818µs ± 0.527µs 505.822µs ± 0.270µs 506.084µs 506.569µs 507.049µs 509.689µs 0.76% 1.823 13.798 0.10% 0.037µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1961980.951op/s 1976997.492op/s ± 2054.720op/s 1976978.307op/s ± 1054.213op/s 1978058.061op/s 1980050.454op/s 1981184.549op/s 1983053.923op/s 0.31% -1.784 13.490 0.10% 145.291op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 452.519µs 453.515µs ± 0.348µs 453.505µs ± 0.239µs 453.770µs 454.060µs 454.308µs 454.706µs 0.26% 0.096 0.370 0.08% 0.025µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2199223.960op/s 2204997.961op/s ± 1692.811op/s 2205045.287op/s ± 1160.376op/s 2206129.975op/s 2207746.702op/s 2208948.999op/s 2209850.741op/s 0.22% -0.091 0.367 0.08% 119.700op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 174.976µs 176.456µs ± 0.499µs 176.518µs ± 0.338µs 176.808µs 177.182µs 177.257µs 177.612µs 0.62% -0.386 -0.401 0.28% 0.035µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5630247.944op/s 5667192.568op/s ± 16038.830op/s 5665152.273op/s ± 10876.622op/s 5677837.289op/s 5695435.334op/s 5702421.264op/s 5715074.939op/s 0.88% 0.398 -0.389 0.28% 1134.117op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.539µs 37.646µs ± 0.041µs 37.642µs ± 0.029µs 37.675µs 37.708µs 37.751µs 37.776µs 0.35% 0.209 0.286 0.11% 0.003µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26472143.610op/s 26563041.670op/s ± 28876.888op/s 26565956.977op/s ± 20129.385op/s 26584229.796op/s 26603665.893op/s 26630677.609op/s 26638698.099op/s 0.27% -0.202 0.281 0.11% 2041.904op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.099µs 48.320µs ± 0.214µs 48.265µs ± 0.159µs 48.543µs 48.660µs 48.699µs 48.716µs 0.93% 0.354 -1.507 0.44% 0.015µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20527285.048op/s 20695695.928op/s ± 91605.902op/s 20718801.307op/s ± 68510.119op/s 20785684.009op/s 20788549.430op/s 20789812.232op/s 20790652.523op/s 0.35% -0.349 -1.513 0.44% 6477.515op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [505.745µs; 505.891µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1976712.728op/s; 1977282.257op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.467µs; 453.564µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2204763.354op/s; 2205232.569op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [176.386µs; 176.525µs] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5664969.741op/s; 5669415.396op/s] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.641µs; 37.652µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26559039.612op/s; 26567043.729op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.290µs; 48.350µs] or [-0.061%; +0.061%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20683000.231op/s; 20708391.625op/s] or [-0.061%; +0.061%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.501µs 209.046µs ± 0.212µs 209.039µs ± 0.115µs 209.146µs 209.309µs 209.733µs 210.675µs 0.78% 2.556 17.355 0.10% 0.015µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4746652.061op/s 4783648.909op/s ± 4846.780op/s 4783787.108op/s ± 2635.364op/s 4786611.688op/s 4789370.089op/s 4792827.877op/s 4796133.677op/s 0.26% -2.517 17.013 0.10% 342.719op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.251µs 18.330µs ± 0.041µs 18.323µs ± 0.027µs 18.353µs 18.386µs 18.410µs 18.662µs 1.85% 2.653 19.045 0.23% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 53584197.435op/s 54556124.607op/s ± 122784.703op/s 54575464.799op/s ± 80912.077op/s 54630685.095op/s 54709508.277op/s 54773941.143op/s 54790357.818op/s 0.39% -2.560 18.109 0.22% 8682.190op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.679µs 10.744µs ± 0.037µs 10.743µs ± 0.028µs 10.768µs 10.802µs 10.834µs 10.863µs 1.12% 0.400 -0.314 0.34% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 92053450.482op/s 93078946.924op/s ± 316299.142op/s 93080302.943op/s ± 247445.112op/s 93353600.669op/s 93538955.002op/s 93578968.244op/s 93643885.270op/s 0.61% -0.384 -0.344 0.34% 22365.727op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [209.016µs; 209.075µs] or [-0.014%; +0.014%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4782977.192op/s; 4784320.626op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.324µs; 18.336µs] or [-0.031%; +0.031%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54539107.828op/s; 54573141.386op/s] or [-0.031%; +0.031%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.739µs; 10.749µs] or [-0.047%; +0.047%] None None None
normalization/normalize_name/normalize_name/good throughput [93035110.905op/s; 93122782.943op/s] or [-0.047%; +0.047%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.052ms 6.064ms ± 0.014ms 6.063ms ± 0.003ms 6.066ms 6.069ms 6.107ms 6.239ms 2.90% 9.660 108.738 0.24% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.062ms; 6.066ms] or [-0.033%; +0.033%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 73ac3d5 1742907933 dsn/r_and_d_week_mar_2024
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.650µs 33.372µs ± 0.902µs 32.820µs ± 0.086µs 34.380µs 35.044µs 35.118µs 35.686µs 8.73% 1.003 -0.776 2.70% 0.064µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.247µs; 33.497µs] or [-0.375%; +0.375%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Mar 11, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 20.19417% with 411 lines in your changes missing coverage. Please review.

Project coverage is 72.35%. Comparing base (51d4c2b) to head (f35e2df).
Report is 2 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #917      +/-   ##
==========================================
- Coverage   72.96%   72.35%   -0.61%     
==========================================
  Files         334      338       +4     
  Lines       50915    51431     +516     
==========================================
+ Hits        37149    37213      +64     
- Misses      13766    14218     +452
Components Coverage Δ
crashtracker 42.88% <ø> (+0.02%) ⬆️
crashtracker-ffi 6.25% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.81% <ø> (-0.16%) ⬇️
data-pipeline-ffi 90.29% <ø> (ø)
ddcommon 81.67% <58.27%> (-1.28%) ⬇️
ddcommon-ffi 68.64% <58.27%> (-1.49%) ⬇️
ddtelemetry 61.87% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.60% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
ipc 82.51% <ø> (+0.10%) ⬆️
profiling 77.60% <6.11%> (-4.18%) ⬇️
profiling-ffi 62.09% <2.35%> (-7.82%) ⬇️
serverless 0.00% <ø> (ø)
sidecar 40.85% <ø> (-1.22%) ⬇️
sidecar-ffi 0.29% <ø> (-9.56%) ⬇️
spawn-worker 54.37% <ø> (ø)
tinybytes 91.59% <ø> (ø)
trace-mini-agent 74.66% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 96.00% <ø> (ø)
trace-protobuf 78.13% <ø> (ø)
trace-utils 92.91% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

taegyunkim
taegyunkim reviewed Mar 11, 2025
View reviewed changes
Copy link
Contributor

@taegyunkim taegyunkim left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reminded me of @ivoanjo 's ManagedStringStorage.
Correct me if I understood it wrong.

  • this change uses strings: StringTable where ivo's PR uses string_storage: Option<...<...<ManagedStringStorage>>> from internal::Profile.
  • we create a new StringTable for each Profile, while string_storage persists across Profiles

morrisonlevi
morrisonlevi reviewed Mar 13, 2025
View reviewed changes
Copy link
Contributor

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to jump to other things, but figured I'd leave my partial review.

@ivoanjo ivoanjo mentioned this pull request Mar 13, 2025
Draft
@danielsn danielsn force-pushed the dsn/r_and_d_week_mar_2024 branch from 8145a59 to 33380b9 Compare March 13, 2025 17:36
@r1viollet
Copy link
Contributor

r1viollet commented Mar 20, 2025 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.83 MB 23.90 MB +.26% (+65.34 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 77.23 MB 77.37 MB +.17% (+139.33 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.69 MB 7.76 MB +.89% (+70.85 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 71.54 MB 71.67 MB +.19% (+140.51 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.42 MB 22.48 MB +.28% (+65.89 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 7.62 MB 7.63 MB +.08% (+6.86 KB) 🔍
i686-alpine-linux-musl
Artifact Baseline Commit Change
/i686-alpine-linux-musl/lib/libdatadog_profiling.a 66.56 MB 66.69 MB +.19% (+129.54 KB) 🔍
/i686-alpine-linux-musl/lib/libdatadog_profiling.so 8.19 MB 8.20 MB +.14% (+12.45 KB) 🔍
/i686-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.01 MB 23.07 MB +.26% (+61.67 KB) 🔍
i686-unknown-linux-gnu
Artifact Baseline Commit Change
/i686-unknown-linux-gnu/lib/libdatadog_profiling.a 67.45 MB 67.58 MB +.18% (+128.76 KB) 🔍
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so 8.07 MB 8.08 MB +.14% (+12.32 KB) 🔍
/i686-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.62 MB 20.68 MB +.28% (+60.78 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 16.59 MB 16.64 MB +.31% (+53.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 55.10 KB 61.18 KB +11.04% (+6.08 KB) 🚨
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 111.21 MB 111.70 MB +.44% (+504.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 625.62 MB 627.46 MB +.29% (+1.83 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 4.99 MB 5.00 MB +.20% (+10.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 55.10 KB 61.18 KB +11.04% (+6.08 KB) 🚨
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 15.85 MB 15.90 MB +.29% (+48.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.38 MB 26.45 MB +.27% (+75.45 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.03 MB 14.08 MB +.35% (+51.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 55.94 KB 62.13 KB +11.05% (+6.18 KB) 🚨
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 113.19 MB 113.67 MB +.42% (+496.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 618.14 MB 619.15 MB +.16% (+1.00 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.78 MB 3.79 MB +.28% (+11.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 55.94 KB 62.13 KB +11.05% (+6.18 KB) 🚨
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.51 MB 16.56 MB +.28% (+48.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.36 MB 24.43 MB +.30% (+76.76 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 66.56 MB 66.69 MB +.19% (+129.54 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.19 MB 8.20 MB +.14% (+12.45 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.01 MB 23.07 MB +.26% (+61.67 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 67.45 MB 67.58 MB +.18% (+128.76 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.07 MB 8.08 MB +.14% (+12.32 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.62 MB 20.68 MB +.28% (+60.78 KB) 🔍

taegyunkim
taegyunkim reviewed Mar 20, 2025
View reviewed changes
@danielsn danielsn enabled auto-merge (squash) March 21, 2025 01:00
@danielsn danielsn disabled auto-merge March 25, 2025 13:31
@danielsn danielsn enabled auto-merge (squash) March 25, 2025 13:31
@danielsn danielsn merged commit fff4bea into main Mar 25, 2025
34 checks passed
@danielsn danielsn deleted the dsn/r_and_d_week_mar_2024 branch March 25, 2025 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrisonlevi morrisonlevi morrisonlevi left review comments

@ivoanjo ivoanjo ivoanjo left review comments

@taegyunkim taegyunkim taegyunkim approved these changes

Assignees
No one assigned
Labels
common profiling Relates to the profiling* modules.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@danielsn @codecov-commenter @r1viollet @morrisonlevi @ivoanjo @taegyunkim