Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemetry client implementation to data pipeline. #802

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

hoolioh
Copy link
Contributor

@hoolioh hoolioh commented Dec 16, 2024

What does this PR do?

Add a client in data pipeline to handle metrics based on SendDataResults.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@pr-commenter
Copy link

pr-commenter bot commented Dec 16, 2024

Benchmarks

Comparison

Benchmark execution time: 2025-01-09 14:01:29

Comparing candidate commit 2846c2d in PR branch julio/APMSP-1449-add-telemetry-metrics with baseline commit 9d10938 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 544.431µs 545.842µs ± 1.066µs 545.642µs ± 0.455µs 546.190µs 547.168µs 549.925µs 554.044µs 1.54% 4.014 24.396 0.19% 0.075µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1804910.801op/s 1832037.313op/s ± 3551.554op/s 1832702.969op/s ± 1528.899op/s 1833989.242op/s 1835343.501op/s 1836330.916op/s 1836781.738op/s 0.22% -3.954 23.809 0.19% 251.133op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 475.149µs 475.990µs ± 0.511µs 475.924µs ± 0.184µs 476.120µs 476.504µs 476.828µs 480.254µs 0.91% 5.139 39.115 0.11% 0.036µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2082230.345op/s 2100886.165op/s ± 2241.268op/s 2101175.469op/s ± 810.841op/s 2101870.936op/s 2103174.129op/s 2103941.255op/s 2104605.075op/s 0.16% -5.092 38.623 0.11% 158.482op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 194.594µs 195.070µs ± 0.523µs 195.011µs ± 0.130µs 195.148µs 195.428µs 195.552µs 200.071µs 2.59% 7.944 71.088 0.27% 0.037µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 4998233.519op/s 5126409.799op/s ± 13454.957op/s 5127911.017op/s ± 3407.029op/s 5131193.651op/s 5135714.708op/s 5137032.461op/s 5138904.591op/s 0.21% -7.864 70.099 0.26% 951.409op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 47.076µs 47.720µs ± 0.212µs 47.727µs ± 0.143µs 47.858µs 48.071µs 48.190µs 48.341µs 1.29% -0.043 0.109 0.44% 0.015µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 20686556.841op/s 20955781.298op/s ± 93167.181op/s 20952693.207op/s ± 62832.034op/s 21025911.351op/s 21105858.752op/s 21178385.007op/s 21242334.865op/s 1.38% 0.071 0.115 0.44% 6587.915op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.909µs 50.164µs ± 0.110µs 50.178µs ± 0.089µs 50.260µs 50.326µs 50.365µs 50.380µs 0.40% -0.185 -0.896 0.22% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19849146.487op/s 19934845.462op/s ± 43656.806op/s 19929029.225op/s ± 35217.709op/s 19972039.145op/s 20003277.037op/s 20024442.206op/s 20036626.954op/s 0.54% 0.192 -0.893 0.22% 3087.002op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [545.695µs; 545.990µs] or [-0.027%; +0.027%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1831545.102op/s; 1832529.525op/s] or [-0.027%; +0.027%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [475.919µs; 476.061µs] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2100575.547op/s; 2101196.783op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [194.997µs; 195.142µs] or [-0.037%; +0.037%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5124545.071op/s; 5128274.526op/s] or [-0.036%; +0.036%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [47.691µs; 47.750µs] or [-0.062%; +0.062%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [20942869.223op/s; 20968693.373op/s] or [-0.062%; +0.062%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [50.148µs; 50.179µs] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19928795.049op/s; 19940895.876op/s] or [-0.030%; +0.030%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 296.569µs 300.573µs ± 1.528µs 300.572µs ± 0.926µs 301.406µs 302.861µs 303.944µs 309.808µs 3.07% 1.521 7.671 0.51% 0.108µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3227800.978op/s 3327062.050op/s ± 16791.496op/s 3326987.618op/s ± 10283.283op/s 3338081.035op/s 3350127.857op/s 3357122.237op/s 3371898.696op/s 1.35% -1.412 7.003 0.50% 1187.338op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.026µs 27.354µs ± 0.112µs 27.352µs ± 0.060µs 27.419µs 27.487µs 27.570µs 28.231µs 3.21% 2.184 17.586 0.41% 0.008µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 35421587.737op/s 36557959.939op/s ± 148613.244op/s 36560391.478op/s ± 80721.216op/s 36640089.807op/s 36753122.734op/s 36899467.379op/s 37001666.837op/s 1.21% -2.008 16.043 0.41% 10508.543op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 15.890µs 16.090µs ± 0.160µs 16.014µs ± 0.057µs 16.249µs 16.388µs 16.430µs 16.468µs 2.83% 0.806 -0.890 0.99% 0.011µs 1 200
normalization/normalize_name/normalize_name/good throughput 60723265.553op/s 62156471.368op/s ± 613510.339op/s 62443757.639op/s ± 224247.901op/s 62607606.848op/s 62792950.069op/s 62867200.804op/s 62932494.233op/s 0.78% -0.792 -0.915 0.98% 43381.732op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [300.361µs; 300.785µs] or [-0.070%; +0.070%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3324734.910op/s; 3329389.190op/s] or [-0.070%; +0.070%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [27.339µs; 27.370µs] or [-0.057%; +0.057%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [36537363.573op/s; 36578556.306op/s] or [-0.056%; +0.056%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.068µs; 16.112µs] or [-0.138%; +0.138%] None None None
normalization/normalize_name/normalize_name/good throughput [62071444.736op/s; 62241498.001op/s] or [-0.137%; +0.137%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.218ms 6.232ms ± 0.013ms 6.231ms ± 0.004ms 6.236ms 6.244ms 6.250ms 6.389ms 2.53% 8.733 100.156 0.21% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.231ms; 6.234ms] or [-0.029%; +0.029%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 58.020ms 58.402ms ± 0.188ms 58.381ms ± 0.087ms 58.469ms 58.689ms 58.921ms 59.758ms 2.36% 2.391 13.243 0.32% 0.013ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [58.376ms; 58.428ms] or [-0.045%; +0.045%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 77.536µs 77.804µs ± 0.139µs 77.789µs ± 0.049µs 77.841µs 77.916µs 78.206µs 79.311µs 1.96% 6.733 68.555 0.18% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [77.785µs; 77.823µs] or [-0.025%; +0.025%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.808µs 2.835µs ± 0.014µs 2.833µs ± 0.009µs 2.843µs 2.858µs 2.866µs 2.882µs 1.72% 0.515 -0.213 0.49% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.833µs; 2.837µs] or [-0.068%; +0.068%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 4.274µs 4.291µs ± 0.005µs 4.291µs ± 0.001µs 4.292µs 4.296µs 4.298µs 4.348µs 1.33% 7.755 91.344 0.11% 0.000µs 1 200
credit_card/is_card_number/ throughput 229978433.726op/s 233024014.119op/s ± 262386.787op/s 233046306.132op/s ± 67167.827op/s 233111999.440op/s 233249581.791op/s 233378967.353op/s 233967856.972op/s 0.40% -7.643 89.776 0.11% 18553.548op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 92.497µs 93.574µs ± 0.520µs 93.534µs ± 0.228µs 93.786µs 94.147µs 94.392µs 98.853µs 5.69% 5.141 51.506 0.55% 0.037µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 10116032.787op/s 10687038.181op/s ± 57776.537op/s 10691333.620op/s ± 26089.159op/s 10712581.940op/s 10764054.871op/s 10788363.592op/s 10811158.166op/s 1.12% -4.707 46.005 0.54% 4085.418op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 87.127µs 87.973µs ± 0.403µs 87.945µs ± 0.157µs 88.110µs 88.393µs 88.584µs 92.436µs 5.11% 6.798 73.335 0.46% 0.029µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 10818252.784op/s 11367292.570op/s ± 50561.555op/s 11370695.444op/s ± 20396.722op/s 11391026.434op/s 11421243.574op/s 11436100.292op/s 11477487.964op/s 0.94% -6.407 67.742 0.44% 3575.242op/s 1 200
credit_card/is_card_number/37828224631 execution_time 4.274µs 4.292µs ± 0.005µs 4.291µs ± 0.001µs 4.293µs 4.295µs 4.297µs 4.351µs 1.40% 8.344 100.931 0.12% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 229828007.604op/s 233018255.323op/s ± 268122.127op/s 233043778.191op/s ± 67290.675op/s 233101182.992op/s 233233845.961op/s 233377707.372op/s 233992165.328op/s 0.41% -8.228 99.297 0.11% 18959.097op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 85.161µs 85.645µs ± 0.209µs 85.640µs ± 0.147µs 85.788µs 86.022µs 86.166µs 86.247µs 0.71% 0.209 -0.019 0.24% 0.015µs 1 200
credit_card/is_card_number/378282246310005 throughput 11594587.805op/s 11676144.197op/s ± 28527.547op/s 11676780.723op/s ± 20010.566op/s 11696655.926op/s 11720445.263op/s 11739150.643op/s 11742504.343op/s 0.56% -0.195 -0.029 0.24% 2017.202op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 58.964µs 59.081µs ± 0.127µs 59.070µs ± 0.015µs 59.085µs 59.129µs 59.203µs 60.389µs 2.23% 8.941 84.477 0.21% 0.009µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 16559378.131op/s 16926051.801op/s ± 35666.735op/s 16929175.103op/s ± 4406.028op/s 16933847.923op/s 16946278.264op/s 16955399.840op/s 16959534.551op/s 0.18% -8.899 83.903 0.21% 2522.019op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.833µs 6.842µs ± 0.004µs 6.841µs ± 0.002µs 6.843µs 6.848µs 6.851µs 6.858µs 0.24% 0.416 2.038 0.05% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 145818204.196op/s 146163567.847op/s ± 79560.875op/s 146167572.724op/s ± 36848.372op/s 146203168.954op/s 146310255.019op/s 146340972.748op/s 146346214.503op/s 0.12% -0.410 2.023 0.05% 5625.803op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 4.279µs 4.291µs ± 0.002µs 4.291µs ± 0.001µs 4.292µs 4.294µs 4.297µs 4.298µs 0.16% -0.571 3.513 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 232678967.408op/s 233056828.898op/s ± 130300.658op/s 233059890.394op/s ± 65378.657op/s 233116487.164op/s 233271383.319op/s 233382152.525op/s 233724863.422op/s 0.29% 0.580 3.541 0.06% 9213.648op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 69.922µs 71.340µs ± 0.452µs 71.374µs ± 0.281µs 71.636µs 71.976µs 72.355µs 72.677µs 1.83% -0.170 0.202 0.63% 0.032µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 13759532.899op/s 14018037.642op/s ± 88842.887op/s 14010752.620op/s ± 54991.677op/s 14071599.497op/s 14170361.742op/s 14226618.566op/s 14301549.604op/s 2.08% 0.211 0.206 0.63% 6282.141op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 64.815µs 66.146µs ± 0.368µs 66.116µs ± 0.266µs 66.418µs 66.750µs 66.952µs 66.978µs 1.30% -0.033 0.057 0.56% 0.026µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 14930305.138op/s 15118614.021op/s ± 84204.915op/s 15124864.537op/s ± 61083.373op/s 15176796.565op/s 15248372.843op/s 15270974.776op/s 15428481.631op/s 2.01% 0.068 0.108 0.56% 5954.187op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 4.276µs 4.291µs ± 0.003µs 4.291µs ± 0.001µs 4.292µs 4.295µs 4.297µs 4.298µs 0.17% -0.670 4.568 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 232680616.457op/s 233067275.350op/s ± 144033.718op/s 233070201.975op/s ± 62664.689op/s 233128966.275op/s 233275332.085op/s 233406701.626op/s 233872809.311op/s 0.34% 0.681 4.618 0.06% 10184.722op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 62.796µs 63.413µs ± 0.281µs 63.400µs ± 0.209µs 63.632µs 63.933µs 64.114µs 64.223µs 1.30% 0.336 -0.302 0.44% 0.020µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 15570839.287op/s 15769989.960op/s ± 69808.024op/s 15772958.075op/s ± 52086.643op/s 15822562.220op/s 15872541.722op/s 15902582.342op/s 15924685.510op/s 0.96% -0.315 -0.325 0.44% 4936.173op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 58.972µs 59.067µs ± 0.041µs 59.062µs ± 0.019µs 59.091µs 59.139µs 59.174µs 59.189µs 0.22% 0.340 0.338 0.07% 0.003µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 16894908.600op/s 16929816.208op/s ± 11620.060op/s 16931305.461op/s ± 5427.267op/s 16936103.469op/s 16948520.060op/s 16954427.400op/s 16957088.392op/s 0.15% -0.335 0.335 0.07% 821.662op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.833µs 6.843µs ± 0.004µs 6.842µs ± 0.002µs 6.845µs 6.850µs 6.855µs 6.861µs 0.27% 0.643 2.119 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 145755416.828op/s 146135550.117op/s ± 86933.528op/s 146146320.459op/s ± 44850.559op/s 146184307.115op/s 146276243.056op/s 146337575.469op/s 146353290.436op/s 0.14% -0.636 2.103 0.06% 6147.129op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [4.291µs; 4.292µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/ throughput [232987649.834op/s; 233060378.404op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [93.502µs; 93.646µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [10679030.908op/s; 10695045.453op/s] or [-0.075%; +0.075%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [87.918µs; 88.029µs] or [-0.064%; +0.064%] None None None
credit_card/is_card_number/ 378282246310005 throughput [11360285.225op/s; 11374299.915op/s] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/37828224631 execution_time [4.291µs; 4.292µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/37828224631 throughput [232981096.175op/s; 233055414.471op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/378282246310005 execution_time [85.616µs; 85.674µs] or [-0.034%; +0.034%] None None None
credit_card/is_card_number/378282246310005 throughput [11672190.553op/s; 11680097.840op/s] or [-0.034%; +0.034%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [59.063µs; 59.098µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [16921108.734op/s; 16930994.867op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.841µs; 6.842µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/x371413321323331 throughput [146152541.475op/s; 146174594.219op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ execution_time [4.290µs; 4.291µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ throughput [233038770.480op/s; 233074887.316op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [71.277µs; 71.402µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [14005724.872op/s; 14030350.412op/s] or [-0.088%; +0.088%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [66.095µs; 66.197µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [15106944.030op/s; 15130284.012op/s] or [-0.077%; +0.077%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [4.290µs; 4.291µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [233047313.662op/s; 233087237.038op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [63.374µs; 63.452µs] or [-0.061%; +0.061%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [15760315.239op/s; 15779664.681op/s] or [-0.061%; +0.061%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [59.062µs; 59.073µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [16928205.779op/s; 16931426.636op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.842µs; 6.844µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [146123501.966op/s; 146147598.268op/s] or [-0.008%; +0.008%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 295.955ns 308.607ns ± 14.225ns 302.167ns ± 4.882ns 314.227ns 341.989ns 348.671ns 349.232ns 15.58% 1.427 0.928 4.60% 1.006ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [306.636ns; 310.578ns] or [-0.639%; +0.639%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.394µs 3.347µs ± 1.491µs 3.123µs ± 0.025µs 3.150µs 3.826µs 14.515µs 15.506µs 396.45% 7.401 55.686 44.45% 0.105µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.141µs; 3.554µs] or [-6.175%; +6.175%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.558µs 27.896µs ± 13.752µs 18.808µs ± 0.200µs 36.478µs 46.877µs 57.968µs 147.313µs 683.25% 3.869 28.091 49.17% 0.972µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [25.990µs; 29.802µs] or [-6.832%; +6.832%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 138.102µs 139.382µs ± 0.479µs 139.397µs ± 0.232µs 139.599µs 140.227µs 140.824µs 141.910µs 1.80% 0.909 4.101 0.34% 0.034µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [139.316µs; 139.448µs] or [-0.048%; +0.048%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 2846c2d 1736430623 julio/APMSP-1449-add-telemetry-metrics
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 36.092µs 36.730µs ± 1.018µs 36.257µs ± 0.070µs 36.413µs 38.930µs 38.981µs 39.151µs 7.98% 1.685 0.887 2.76% 0.072µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [36.589µs; 36.872µs] or [-0.384%; +0.384%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Dec 16, 2024

Codecov Report

Attention: Patch coverage is 85.79235% with 52 lines in your changes missing coverage. Please review.

Project coverage is 71.40%. Comparing base (ba2a3fe) to head (2846c2d).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #802      +/-   ##
==========================================
+ Coverage   71.04%   71.40%   +0.36%     
==========================================
  Files         313      315       +2     
  Lines       45898    46281     +383     
==========================================
+ Hits        32606    33045     +439     
+ Misses      13292    13236      -56     
Components Coverage Δ
crashtracker 38.52% <ø> (+0.25%) ⬆️
crashtracker-ffi 5.71% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 91.54% <86.02%> (-0.34%) ⬇️
data-pipeline-ffi 90.54% <ø> (ø)
ddcommon 82.08% <ø> (+0.04%) ⬆️
ddcommon-ffi 65.84% <ø> (+0.24%) ⬆️
ddtelemetry 61.89% <0.00%> (+2.37%) ⬆️
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.59% <ø> (ø)
dogstatsd-client 79.77% <ø> (ø)
ipc 82.69% <ø> (ø)
profiling 84.29% <ø> (ø)
profiling-ffi 77.55% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 41.90% <ø> (+0.99%) ⬆️
sidecar-ffi 11.66% <ø> (+7.68%) ⬆️
spawn-worker 54.37% <ø> (ø)
tinybytes 93.60% <ø> (ø)
trace-mini-agent 72.48% <ø> (ø)
trace-normalization 98.23% <ø> (ø)
trace-obfuscation 95.77% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.79% <ø> (ø)

@hoolioh hoolioh force-pushed the julio/APMSP-1449-add-telemetry-metrics branch from 1e01f44 to f9bc429 Compare December 30, 2024 17:04
@hoolioh hoolioh force-pushed the julio/APMSP-1449-add-telemetry-metrics branch from 7a2c893 to 7513a28 Compare December 31, 2024 09:31
* Create a new builder for the TelemetryClient object.
* Add implementation for the TelemetryClient.
* Add tests.
* Fix pipeline errors.
@hoolioh hoolioh force-pushed the julio/APMSP-1449-add-telemetry-metrics branch from 7513a28 to 95fa3e9 Compare December 31, 2024 09:38
@github-actions github-actions bot removed the sidecar label Dec 31, 2024
@hoolioh hoolioh changed the title Move telemetry client implementation to data pipeline. Add telemetry client implementation to data pipeline. Dec 31, 2024
* Move metrics to another compilation unit.
* Add tests to metrics.
* Refactor builder to use `url` and `heartbeat` properties instead of
  ddtelemetry::config::Config.
* Add basic documentation.
@hoolioh hoolioh force-pushed the julio/APMSP-1449-add-telemetry-metrics branch from 03473ca to ab2a274 Compare December 31, 2024 14:46

impl TelemetryClientBuilder {
/// Creates a new empty builder.
pub fn new() -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel strongly about this, but if we're just returning a default Self, do we need a new() method at all?

data-pipeline/src/lib.rs Outdated Show resolved Hide resolved
}

/// Register a new metric that will be used by the telemetry client.
pub fn add_metric(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the metrics are already pretty much hardcoded in the Metrics struct wouldn't it be easier to hardcode them in the client send function.

@VianneyRuhlmann
Copy link
Contributor

Why can't we rely on ddtelemetry's client aggregation/flush mechanism ? Is it to expensive to add the points on each submitted trace payload ?

hoolioh and others added 2 commits January 9, 2025 14:13
@hoolioh
Copy link
Contributor Author

hoolioh commented Jan 9, 2025

Why can't we rely on ddtelemetry's client aggregation/flush mechanism ? Is it to expensive to add the points on each submitted trace payload ?

The main idea to try if it would make sense to have a common implementation for the sidecar and data-pipeline so we could have more control over trace-utils (SendData and SendDataResult) for further refactor/optimizations. The sidecar was already accumulating the results in a common metric structure so I kind of followed that path. Also the accumulation seemed less expensive than sending data through the telemetry handle on every trace but, honestly, I don't know how much I think we should do some benchmarking to test it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants