Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(sequencer): add benchmark for prepare_proposal (ENG-660) #1329

Closed

Conversation

Fraser999
Copy link
Contributor

@Fraser999 Fraser999 commented Aug 1, 2024

Summary

Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's App.

Background

Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function.

Changes

  • Added benchmark to app module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor.
  • Added benchmark feature to enable sharing some of the existing test utils.

Testing

This is a new test.

Example of running cargo bench --features=benchmark -qp astria-sequencer app on my Ryzen 7900X:

Timer precision: 10 ns
astria_sequencer                                fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ app                                                        │               │               │               │         │
   ╰─ benchmarks                                              │               │               │               │         │
      ╰─ execute_transactions_prepare_proposal  11.63 s       │ 14.68 s       │ 12.74 s       │ 12.88 s       │ 10      │ 10

Since rebasing after #1317 has merged, the same run shows (as expected) a slowdown:

astria_sequencer                                fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ app                                                        │               │               │               │         │
   ╰─ benchmarks                                              │               │               │               │         │
      ╰─ execute_transactions_prepare_proposal  14.49 s       │ 17 s          │ 16.52 s       │ 15.98 s       │ 8       │ 8

Related Issues

Closes #1314.

@Fraser999 Fraser999 requested a review from a team as a code owner August 1, 2024 10:06
@Fraser999 Fraser999 requested a review from SuperFluffy August 1, 2024 10:06
@github-actions github-actions bot added the sequencer pertaining to the astria-sequencer crate label Aug 1, 2024
@Fraser999 Fraser999 force-pushed the fraser/add-sequencer-benchmark branch from 28ff254 to 9b039da Compare August 1, 2024 20:47
@Fraser999
Copy link
Contributor Author

Closed in favour of #1337 (this PR was cut from the wrong repo).

@Fraser999 Fraser999 closed this Aug 1, 2024
@Fraser999 Fraser999 deleted the fraser/add-sequencer-benchmark branch August 1, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sequencer pertaining to the astria-sequencer crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide faster feedback loop for assessing performance of sequencer
1 participant