refactor(sequencer): remove global state #1317
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fraser999
approved these changes
Aug 1, 2024
Comment on lines
+92
to
+99
| let _ = snapshot | ||
| .get_native_asset() | ||
| .await | ||
| .context("failed to get native asset from storage")?; | ||
| crate::assets::initialize_native_asset(&native_asset.to_string()); | ||
| let base_prefix = snapshot | ||
| .context("failed to query state for native asset")?; | ||
| let _ = snapshot | ||
| .get_base_prefix() | ||
| .await | ||
| .context("failed to get address base prefix from storage")?; | ||
| crate::address::initialize_base_prefix(&base_prefix) | ||
| .context("failed to initialize global address base prefix")?; | ||
| .context("failed to query state for base prefix")?; |
There was a problem hiding this comment.
Is there any point to still doing these calls?
There was a problem hiding this comment.
I think you are right. I am going to remove these as part of addressing #1318
This was referenced Aug 1, 2024
steezeburger
added a commit
that referenced
this pull request
Aug 2, 2024
* main: chore(core): Implement Protobuf trait for tx actions (#1320) refactor(sequencer): remove global state (#1317) refactor(sequencer): move asset state methods to asset module (#1313) feat(sequencer, core): Add fee reporting (#1305) chore(bridge-withdrawer): cleanup nonce handling (#1292) fix(charts, bridge): fix ci test (#1310)
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 21, 2024
## Summary
Addition of a new benchmark target to assess the performance of the
prepare_proposal method in sequencer's `App`.
## Background
Previous perf work has indicated this is a bottleneck. However, making
that determination was done via spamoor which is slightly convoluted to
run. Before working to improve the performance, we want to have a faster
feedback loop on the effects of updates, hence the need for a benchmark
which is easy to run and which isolates the slow function.
## Changes
- Added benchmark to `app` module. Currently this has only one case: a
mempool filled with transactions containing exclusively transfers. This
matches the shape of the data being sent when using spamoor.
- Added `benchmark` feature to enable sharing some of the existing test
utils.
## Testing
This is a new test.
Example of running `cargo bench --features=benchmark -qp
astria-sequencer app` on my Ryzen 7900X:
```
Timer precision: 10 ns
astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters
╰─ app │ │ │ │ │
╰─ benchmarks │ │ │ │ │
╰─ execute_transactions_prepare_proposal 11.63 s │ 14.68 s │ 12.74 s │ 12.88 s │ 10 │ 10
```
Since rebasing after #1317 has merged, the same run shows (as expected)
a slowdown:
```
astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters
╰─ app │ │ │ │ │
╰─ benchmarks │ │ │ │ │
╰─ execute_transactions_prepare_proposal 14.49 s │ 17 s │ 16.52 s │ 15.98 s │ 8 │ 8
```
## Related Issues
Closes #1314.
Fraser999
added a commit
to Fraser999/astria
that referenced
this pull request
Aug 23, 2024
…org#1337) ## Summary Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's `App`. ## Background Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function. ## Changes - Added benchmark to `app` module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor. - Added `benchmark` feature to enable sharing some of the existing test utils. ## Testing This is a new test. Example of running `cargo bench --features=benchmark -qp astria-sequencer app` on my Ryzen 7900X: ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 11.63 s │ 14.68 s │ 12.74 s │ 12.88 s │ 10 │ 10 ``` Since rebasing after astriaorg#1317 has merged, the same run shows (as expected) a slowdown: ``` astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 14.49 s │ 17 s │ 16.52 s │ 15.98 s │ 8 │ 8 ``` ## Related Issues Closes astriaorg#1314.
Fraser999
added a commit
to Fraser999/astria
that referenced
this pull request
Aug 25, 2024
…org#1337) ## Summary Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's `App`. ## Background Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function. ## Changes - Added benchmark to `app` module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor. - Added `benchmark` feature to enable sharing some of the existing test utils. ## Testing This is a new test. Example of running `cargo bench --features=benchmark -qp astria-sequencer app` on my Ryzen 7900X: ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 11.63 s │ 14.68 s │ 12.74 s │ 12.88 s │ 10 │ 10 ``` Since rebasing after astriaorg#1317 has merged, the same run shows (as expected) a slowdown: ``` astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 14.49 s │ 17 s │ 16.52 s │ 15.98 s │ 8 │ 8 ``` ## Related Issues Closes astriaorg#1314.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Removed all global from sequencer.
Background
Upon init, sequencer stored its native asset and address base prefix at a fixed global memory locations. This did not yet lead to issues during runtime (because we don't mutate global after startup), it made testing unnecessarily difficult: sequencer relies heavily on unit tests, yet the tests were impure because they relied on the same global state.
The changes to the
mempoolmodule shows another issue: while the mempool could be wholly agnostic to the specific global state, heavy use of the global base prefix meant was less decoupled than it could be.Because sequencer explicitly passes around state (in the form of state deltas, the data inside which are made accessible via various
StateReadExtandStateWriteExttraits), we can read the configured base prefix and native assets from there.The downside is that this patch introduces more database reads. However, given the amounts of data sequencer is dealing with this might not matter and can be alleviated by using a rocksdb Cache (or other mechanism) instead of relying on globals.
Changes
crate::addressandcrate::assetscrate::address::StateReadExtandcrate::address::assets::StateReadExtinstead.TracePrefixedasset.Testing
Updated all tests to either use hard coded prefixes, native assets (if not performing reads or writes on state deltas), or write a base prefix and native asset to state before performing tests (if accessing state).
Breaking Changelist
While the change to
astria-coreis technically breaking (because a field that was of typeasset::Denomis nowasset::TracePrefixed), in practice this change is not breaking because Sequencer required native assets to be non-ibc prefixed.Related Issues
Closes #1208