chore: optimize out retriggering 2nd stage of build + wasmopt stage #292

dj8yfo · 2025-01-16T21:39:46Z

reason for this pr is that LazyLock initialization block
doesn't work to ensure tested contracts are built only once for cargo nextest run as cargo-nextest uses a process-per-test runner,
and for usual cargo test workflows it may not be convenient/possible to unite all tests into a single test module, thus contracts' build will run at least once per each test module.

This pr minimizes time spent on contracts' rebuilding after initial build, so that LazyLock init block isn't that much needed any more, but it depends on separating target folders of different contracts being tested similar to dj8yfo/neardevhub-contract@cbbf0ee to avoid partially rebuilding contracts starting with near-sdk-macros crate, if it's shared in common release profile when different contracts are in a workspace.

this results in

• Building contract
 │     Finished `release` profile [optimized] target(s) in 0.08s

• Running an optimize for size post-step with wasm-opt...
/home/user/Documents/code/near/neardevhub-contract/target/wasm32-unknown-unknown/release/devhub.wasm -> /tmp/optimized-s2bmSK.wasm
• done

✓ Contract successfully built! (in CARGO_NEAR_BUILD_ENVIRONMENT=host)
     -                Binary: /home/user/Documents/code/near/neardevhub-contract/target/near/devhub.wasm
     - SHA-256 checksum hex : 1f568ea4c5eabf9c947a201f3adf38a4e47056658aa512fdfc722f17de9d3aa3
     - SHA-256 checksum bs58: 37LC32jmZFmZN8KesPipwUn4ZyhW4994coXqSWMdnDQv
     -                   ABI: /home/user/Documents/code/near/neardevhub-contract/target/near/devhub_abi.json
     -          Embedded ABI: /home/user/Documents/code/near/neardevhub-contract/target/near/devhub_abi.zst
    Finished cargo near build in 5s
Here is the console command if you ever need to re-run it again:
cargo near build non-reproducible-wasm

instead of

• Building contract
 │    Compiling near-sdk-macros v5.5.0
 │    Compiling near-sdk v5.5.0
 │    Compiling devhub_common v0.1.0 (/home/user/Documents/code/near/neardevhub-contract/d
evhub_common)
 │    Compiling near-contract-standards v5.5.0
 │    Compiling devhub v0.2.0 (/home/user/Documents/code/near/neardevhub-contract)
 │     Finished `release` profile [optimized] target(s) in 15.52s

• Running an optimize for size post-step with wasm-opt...
/home/user/Documents/code/near/neardevhub-contract/target/wasm32-unknown-unknown/release/d
evhub.wasm -> /tmp/optimized-FbzCfP.wasm
• done

✓ Contract successfully built! (in CARGO_NEAR_BUILD_ENVIRONMENT=host)
     -                Binary: /home/user/Documents/code/near/neardevhub-contract/target/ne
ar/devhub.wasm
     - SHA-256 checksum hex : 1f568ea4c5eabf9c947a201f3adf38a4e47056658aa512fdfc722f17de9d3aa3
     - SHA-256 checksum bs58: 37LC32jmZFmZN8KesPipwUn4ZyhW4994coXqSWMdnDQv
     -                   ABI: /home/user/Documents/code/near/neardevhub-contract/target/ne
ar/devhub_abi.json
     -          Embedded ABI: /home/user/Documents/code/near/neardevhub-contract/target/ne
ar/devhub_abi.zst
    Finished cargo near build in 20s

on rebuilds.

and wasm-opt stage skipping was handled in e746699 . No file locks were added, it would still waste some cpu, if multiple builders enter the wasm-opt stage around the same time, but it does save on time and locks are perceived as unneeded complexity at this time.

vaguely related to Near-One/omni-bridge#184

dj8yfo · 2025-01-17T00:36:04Z

adding dj8yfo/neardevhub-contract@cbbf0ee
prevents partial rebuilds of neardevhub-contract and neardevhub-community-factory on

 │        Dirty near-sdk-macros v5.7.0: the environment variable CARGO_NEAR_ABI_PATH changed
 │    Compiling near-sdk-macros v5.7.0

due to near-sdk-macros crate's builds being shared in the same release profile.

Only wasm-opt computing overhead is left per each test with cargo nextest run (5 s for neardevhub and 1 s for community-factory)
Summary [ 204.822s] ?? -> Summary [ 49.310s]

if the abi logic isn't being used/tested in tests, both the reasons of rebuilds can also be eliminated by doing .no_abi(true) build

This reverts commit 9d30422.

This reverts commit e91bd03.

… is newer than the original wasm, which means that we have already optimized it

dj8yfo · 2025-01-20T21:50:35Z

dj8yfo · 2025-01-20T22:52:30Z

running cargo test on clean state (taking into account inital contracts' build) takes 542 s,
and cargo nextest run - 472 s

cargo nextest run from scratch on the same code with unpatched version (from crates.io and not
from this pr) dj8yfo/neardevhub-contract@dd3ada1
of cargo-near-build - 527 s, so the time cut of this pr is 55 s for 7 tests which build contracts.
And 72 s unpatched vs 32 s patched for second run for 7 tests.

dj8yfo · 2025-01-21T20:18:05Z

@race-of-sloths include

race-of-sloths · 2025-01-21T20:19:28Z

@dj8yfo Thank you for your contribution! Your pull request is now a part of the Race of Sloths!
Congratulatory messages are spent out! Your results have exceeded our imagination!

Current status: waiting for finalization

The pull request is merged, you have 24 hours to finalize your scoring. The scoring ends Thu Jan 23 12:48:20 2025

Reviewer	Score
@akorchyn	13

Your contribution is much appreciated with a final score of 13!
You have received 130 Sloth points for this contribution

@akorchyn received 25 Sloth Points for reviewing and scoring this pull request.

What is the Race of Sloths

Race of Sloths is a friendly competition where you can participate in challenges and compete with other open-source contributors within your normal workflow

For contributors:

Tag @race-of-sloths inside your pull requests
Wait for the maintainer to review and score your pull request
Check out your position in the Leaderboard
Keep weekly and monthly streaks to reach higher positions
Boast your contributions with a dynamic picture of your Profile

For maintainers:

Score pull requests that participate in the Race of Sloths and receive a reward
Engage contributors with fair scoring and fast responses so they keep their streaks
Promote the Race to the point where the Race starts promoting you
Grow the community of your contributors

Feel free to check our website for additional details!

Bot commands

For contributors
- Include a PR: @race-of-sloths include to enter the Race with your PR
For maintainers:
- Invite contributor @race-of-sloths invite to invite the contributor to participate in a race or include it, if it's already a runner.
- Assign points: @race-of-sloths score [1/2/3/5/8/13] to award points based on your assessment.
- Reject this PR: @race-of-sloths exclude to send this PR back to the drawing board.
- Exclude repo: @race-of-sloths pause to stop bot activity in this repo until @race-of-sloths unpause command is called

akorchyn

@race-of-sloths score 13

Two nits that you might skip:)

cargo-near-build/src/near/build/mod.rs

## 🤖 New release * `cargo-near`: 0.13.2 -> 0.13.3 (✓ API compatible changes) * `cargo-near-build`: 0.4.2 -> 0.4.3 (✓ API compatible changes) <details><summary>Changelog</summary> ## `cargo-near` <blockquote> ## [0.13.3](cargo-near-v0.13.2...cargo-near-v0.13.3) - 2025-01-22 ### Other - update near-cli-rs to 0.18.0 (#293) - update `cargo near new` template `image` and `image_digest` ([#288](#288)) - update `cargo near new` template `image` and `image_digest` ([#283](#283)) </blockquote> ## `cargo-near-build` <blockquote> ## [0.4.3](cargo-near-build-v0.4.2...cargo-near-build-v0.4.3) - 2025-01-22 ### Fixed - remove from env CARGO_ENCODED_RUSTFLAGS for easier nested builds, simplify RUSTFLAGS computation rule (#289) ### Other - optimize out retriggering 2nd stage of build + of wasmopt stage in tests context (#292) - update `cargo near new` template `image` and `image_digest` ([#288](#288)) - unpin `cc` after issue resolution (#285) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

dj8yf0μl added 3 commits January 16, 2025 23:39

chore: optimize out retriggering 2nd stage of build

342863f

chore: add println!

9d30422

add --verbose

e91bd03

dj8yf0μl added 2 commits January 20, 2025 18:06

Revert "chore: add println!"

73c292c

This reverts commit 9d30422.

Revert "add --verbose"

46d8f62

This reverts commit e91bd03.

dj8yfo force-pushed the shorten_2nd_phase_of_build_on_rebuild_to_fast branch from 8ab53be to 5b68575 Compare January 20, 2025 16:06

dj8yfo mentioned this pull request Jan 20, 2025

make cargo-near-buid::extended::build api ergonomic by using sane defaults #291

Open

8 tasks

dj8yf0μl and others added 2 commits January 20, 2025 19:08

chore: print wasm-opt duration as milliseconds

15d3749

DO NOT MERGE WITHOUT REFACTORING: skip wasm-opt if the optimized wasm…

e746699

… is newer than the original wasm, which means that we have already optimized it

dj8yfo force-pushed the shorten_2nd_phase_of_build_on_rebuild_to_fast branch from 5b68575 to e746699 Compare January 20, 2025 17:09

chore: split logic around is_newer_than; print debug output

77b9074

dj8yfo changed the title ~~chore: optimize out retriggering 2nd stage of build~~ chore: optimize out retriggering 2nd stage of build + wasmopt stage Jan 20, 2025

dj8yfo mentioned this pull request Jan 21, 2025

generalize compile_project helper, add automatic target dir separation per contract near/near-workspaces-rs#401

Open

dj8yfo marked this pull request as ready for review January 21, 2025 20:16

dj8yfo requested review from FroVolod, akorchyn and PolyProgrammist as code owners January 21, 2025 20:16

akorchyn previously approved these changes Jan 22, 2025

View reviewed changes

cargo-near-build/src/near/build/mod.rs Outdated Show resolved Hide resolved

cargo-near-build/src/near/build/mod.rs Outdated Show resolved Hide resolved

dj8yfo and others added 2 commits January 22, 2025 13:17

Merge branch 'main' into shorten_2nd_phase_of_build_on_rebuild_to_fast

ce4305f

chore: fix nit and msg

2057169

dj8yfo dismissed akorchyn’s stale review via 2057169 January 22, 2025 11:56

chore: return removed msg as tracing::debug!

4b067f2

dj8yfo enabled auto-merge (squash) January 22, 2025 12:05

dj8yfo disabled auto-merge January 22, 2025 12:34

akorchyn approved these changes Jan 22, 2025

View reviewed changes

dj8yfo merged commit 2851dc6 into near:main Jan 22, 2025
17 checks passed

frol mentioned this pull request Jan 22, 2025

chore: release #284

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: optimize out retriggering 2nd stage of build + wasmopt stage #292

chore: optimize out retriggering 2nd stage of build + wasmopt stage #292

dj8yfo commented Jan 16, 2025 •

edited

Loading

dj8yfo commented Jan 17, 2025 •

edited

Loading

dj8yfo commented Jan 20, 2025 •

edited

Loading

dj8yfo commented Jan 20, 2025 •

edited

Loading

dj8yfo commented Jan 21, 2025

race-of-sloths commented Jan 21, 2025 •

edited

Loading

akorchyn left a comment

chore: optimize out retriggering 2nd stage of build + wasmopt stage #292

chore: optimize out retriggering 2nd stage of build + wasmopt stage #292

Conversation

dj8yfo commented Jan 16, 2025 • edited Loading

dj8yfo commented Jan 17, 2025 • edited Loading

dj8yfo commented Jan 20, 2025 • edited Loading

dj8yfo commented Jan 20, 2025 • edited Loading

dj8yfo commented Jan 21, 2025

race-of-sloths commented Jan 21, 2025 • edited Loading

akorchyn left a comment

Choose a reason for hiding this comment

dj8yfo commented Jan 16, 2025 •

edited

Loading

dj8yfo commented Jan 17, 2025 •

edited

Loading

dj8yfo commented Jan 20, 2025 •

edited

Loading

dj8yfo commented Jan 20, 2025 •

edited

Loading

race-of-sloths commented Jan 21, 2025 •

edited

Loading