Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ICP-ledger): FI-1438: Implement V3 for ICP ledger - migrate allowances to stable structures #2818

Merged
merged 209 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from 208 commits
Commits
Show all changes
209 commits
Select commit Hold shift + click to select a range
8db3763
v2 version for the icrc ledger
maciejdfinity Sep 10, 2024
619a887
update mainnet canisters to v1
maciejdfinity Sep 11, 2024
0913c4b
Merge branch 'master' into maciej-icrc-v2
maciejdfinity Sep 13, 2024
7f1a685
update mainnet canister hash
maciejdfinity Sep 13, 2024
702e5d8
remove extra clone
maciejdfinity Sep 13, 2024
d89d15d
approvals in stable memory
maciejdfinity Sep 16, 2024
ce95fcc
Automatically updated Cargo*.lock
Sep 16, 2024
df25efd
skip arrived_at serialization
maciejdfinity Sep 18, 2024
b262c9a
comment
maciejdfinity Sep 18, 2024
2ba7c8f
migrate approvals
maciejdfinity Sep 18, 2024
af7376e
Automatically updated Cargo*.lock
Sep 18, 2024
0fc1d35
disable ledger while migration and wait for ledger to be ready
maciejdfinity Sep 23, 2024
823472c
remove approval trimming test
maciejdfinity Sep 24, 2024
871d1e1
clippy
maciejdfinity Sep 24, 2024
ab60f40
build fix
maciejdfinity Sep 24, 2024
f108662
fix candid
maciejdfinity Sep 24, 2024
e454fbc
fix tests
maciejdfinity Sep 24, 2024
378c316
fix tests
maciejdfinity Sep 24, 2024
2b64a03
clippy
maciejdfinity Sep 24, 2024
9f545df
fix tests
maciejdfinity Sep 24, 2024
82ff283
clippy
maciejdfinity Sep 24, 2024
3171aa0
test account maxserialization len
maciejdfinity Sep 24, 2024
ed7731d
test account serialization
maciejdfinity Sep 24, 2024
5585594
test allowance serialization
maciejdfinity Sep 24, 2024
eb830db
account for pre_upgrade instructions
maciejdfinity Sep 25, 2024
247d2e3
additional ready checks
maciejdfinity Sep 25, 2024
dbc7f32
remove is_migrating
maciejdfinity Sep 25, 2024
9559981
migration steps metric
maciejdfinity Sep 25, 2024
35924ed
clippy
maciejdfinity Sep 25, 2024
5b7c9a3
update limits, do not write in pre_upgrade if not ready
maciejdfinity Sep 26, 2024
11324d7
clippy
maciejdfinity Sep 27, 2024
7434304
require memory manager
maciejdfinity Sep 30, 2024
1ae7323
test endpoints disabled during migration and test failed migration
maciejdfinity Sep 30, 2024
9e50e99
clippy
maciejdfinity Sep 30, 2024
8da91b7
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Oct 1, 2024
0fa6ce9
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Oct 7, 2024
1086b5a
build fix
maciejdfinity Oct 7, 2024
8811688
clippy
maciejdfinity Oct 7, 2024
ac7ec29
remove test upgrade from first version
maciejdfinity Oct 7, 2024
c8b94d1
feature flag with lower instuction limits
maciejdfinity Oct 8, 2024
9e67753
restore mainnet canisters
maciejdfinity Oct 11, 2024
e058e16
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Oct 11, 2024
d8ca322
update mainnet canisters json
maciejdfinity Oct 11, 2024
43efb21
implement stable structures storable for account
maciejdfinity Oct 11, 2024
1a03ada
clippy
maciejdfinity Oct 11, 2024
cef0b73
Automatically updated Cargo*.lock
Oct 11, 2024
168afff
implement storable for allowance
maciejdfinity Oct 11, 2024
aff2fcd
Automatically updated Cargo*.lock
Oct 11, 2024
b8ff51d
Merge branch 'master' into maciej-storable-account
maciejdfinity Oct 11, 2024
9ef97c2
Merge branch 'master' into maciej-storable-account
maciejdfinity Oct 13, 2024
0b5dfa2
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Oct 13, 2024
193b566
increase max allowed orchestrator wasm size
maciejdfinity Oct 14, 2024
bbd77ad
fix test upgrade from 256 to 64
maciejdfinity Oct 14, 2024
f72145b
update canbench, pre_upgrade improved
maciejdfinity Oct 14, 2024
356f8b8
refactor
maciejdfinity Oct 14, 2024
e1eccd4
test multiple round upgrade
maciejdfinity Oct 14, 2024
73f6240
panic if clearing stable allowance is called
maciejdfinity Oct 14, 2024
6308d5f
update ledger and archive wasms
maciejdfinity Oct 15, 2024
3109c06
readd approval trimming test
maciejdfinity Oct 15, 2024
d854423
remove unused param
maciejdfinity Oct 15, 2024
1de5b1d
Update rs/ledger_suite/common/ledger_core/src/approvals.rs
maciejdfinity Oct 15, 2024
6b1f3db
storable for account id
maciejdfinity Oct 16, 2024
c11bd8c
move dep
maciejdfinity Oct 16, 2024
ff05761
fix to bytes
maciejdfinity Oct 16, 2024
6879d08
fix serialization
maciejdfinity Oct 16, 2024
dbffb1f
adding StableAllowancesData
maciejdfinity Oct 16, 2024
455658f
ledger state and migration code
maciejdfinity Oct 16, 2024
29958d5
migration steps counter
maciejdfinity Oct 16, 2024
7b30bd9
use ledger context methods for accessing approvals and balances
maciejdfinity Oct 16, 2024
2c69e72
fix deps
maciejdfinity Oct 17, 2024
974b8f5
remove approvals trimming
maciejdfinity Oct 17, 2024
b957259
clippy
maciejdfinity Oct 17, 2024
b3f88e0
only disable endpoints that change ledger state
maciejdfinity Oct 17, 2024
ccb5710
proptest account serialization
maciejdfinity Oct 17, 2024
b69eb98
proptest allowance serialization
maciejdfinity Oct 17, 2024
8873fb8
Automatically updated Cargo*.lock
Oct 17, 2024
3b3dd3d
Merge branch 'maciej-storable-allowance' into maciej-icrc-v3
maciejdfinity Oct 17, 2024
2974e2d
Merge branch 'maciej-storable-account' into maciej-icrc-v3
maciejdfinity Oct 17, 2024
9a28bc4
migrate code in ledger
maciejdfinity Oct 17, 2024
e5ff866
Merge branch 'master' into maciej-storable-account
maciejdfinity Oct 17, 2024
d0f82cf
Merge branch 'maciej-storable-account' into maciej-storable-allowance
maciejdfinity Oct 17, 2024
8d3bf63
Merge branch 'maciej-storable-allowance' into maciej-icrc-v3
maciejdfinity Oct 17, 2024
a24fe45
clippy
maciejdfinity Oct 17, 2024
accaa54
fix migration to prev version tests
maciejdfinity Oct 17, 2024
a8035e3
build fix
maciejdfinity Oct 17, 2024
c07c8ac
disable downgrade to mainnet test
maciejdfinity Oct 17, 2024
56940aa
fix test
maciejdfinity Oct 17, 2024
48e0fe4
fix test
maciejdfinity Oct 17, 2024
7f3e024
update canbench results
maciejdfinity Oct 17, 2024
85c0d9d
larger limits
maciejdfinity Oct 18, 2024
095509e
move state below version
maciejdfinity Oct 18, 2024
0a29200
move test up in file
maciejdfinity Oct 18, 2024
692f896
Update rs/ledger_suite/icrc1/ledger/src/lib.rs
maciejdfinity Nov 4, 2024
633653f
Update rs/ledger_suite/icrc1/ledger/src/lib.rs
maciejdfinity Nov 4, 2024
fc0b00d
Update rs/ledger_suite/icrc1/ledger/src/lib.rs
maciejdfinity Nov 4, 2024
1d06a25
Update rs/ledger_suite/tests/sm-tests/src/lib.rs
maciejdfinity Nov 4, 2024
0203191
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 4, 2024
0c2e9c3
extend the comment for ledger_version
maciejdfinity Nov 4, 2024
aba9408
panic if not ready fn
maciejdfinity Nov 4, 2024
0cd743f
build fix
maciejdfinity Nov 4, 2024
d644f2f
comment about not storing arrivals
maciejdfinity Nov 4, 2024
00c5d40
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 4, 2024
ebf3c2a
improve logging
maciejdfinity Nov 4, 2024
5ad210d
update comment
maciejdfinity Nov 4, 2024
27b06e7
test failure to downgrade from stable structures
maciejdfinity Nov 4, 2024
5fe793c
assert ledger not ready
maciejdfinity Nov 4, 2024
5c2757a
remove unused tests
maciejdfinity Nov 4, 2024
752266c
remove first version icrc wasm
maciejdfinity Nov 4, 2024
4386237
clippy
maciejdfinity Nov 4, 2024
3f4e287
check for ready in endpoints
maciejdfinity Nov 5, 2024
6d48c28
fix test
maciejdfinity Nov 5, 2024
9127d6e
update canbench results
maciejdfinity Nov 5, 2024
cfd6493
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 6, 2024
6fed6f3
test metrics while updating
maciejdfinity Nov 6, 2024
d96eac5
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 6, 2024
8afc10d
log migration step instructions
maciejdfinity Nov 6, 2024
8665695
test timestamp serialization
maciejdfinity Nov 6, 2024
6953942
use thread local for ledger state
maciejdfinity Nov 7, 2024
d3f51c4
clippy
maciejdfinity Nov 7, 2024
a7fd4ab
update canbench results
maciejdfinity Nov 7, 2024
b33b6a0
migrate some allowances in post_upgrade
maciejdfinity Nov 8, 2024
a4fa643
speed up account serialization and deserialization
maciejdfinity Nov 11, 2024
b9ed933
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 11, 2024
2730a5c
simplify the code a bit
maciejdfinity Nov 12, 2024
186b892
migrating with interval timer
maciejdfinity Nov 12, 2024
a5ec43d
store allowance expiration length while serializing
maciejdfinity Nov 15, 2024
c93ecbb
test if migration resumes when frozen and unfrozen
maciejdfinity Nov 18, 2024
cfa4744
Revert "migrating with interval timer"
maciejdfinity Nov 18, 2024
dafc6c7
make sure the timer was attempted to be scheduled
maciejdfinity Nov 19, 2024
40b015c
serializa Account with minicbor
maciejdfinity Nov 19, 2024
1e349d4
fix error message
maciejdfinity Nov 19, 2024
50119ed
clippy
maciejdfinity Nov 19, 2024
70ed4ae
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 19, 2024
b661488
clippy
maciejdfinity Nov 19, 2024
ead1399
clippy
maciejdfinity Nov 19, 2024
16a64ee
clippy
maciejdfinity Nov 19, 2024
4be7e56
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 20, 2024
4c97ac5
build fix
maciejdfinity Nov 20, 2024
d538875
use minicbor for allowance serialization
maciejdfinity Nov 20, 2024
a4eeb3e
clippy
maciejdfinity Nov 20, 2024
f1af825
clippy
maciejdfinity Nov 20, 2024
68565e4
Merge branch 'maciej-icrc-v3' into maciej-icp-v3
maciejdfinity Nov 20, 2024
f884cea
storable allowance
maciejdfinity Nov 20, 2024
13da037
rename decoder
maciejdfinity Nov 20, 2024
1501fa5
comment about arrived_at unused
maciejdfinity Nov 20, 2024
fe34d6d
clear stable allowances in case of incomplete migration
maciejdfinity Nov 20, 2024
e1a72b0
simplify clearing stable allowances
maciejdfinity Nov 21, 2024
8867fa3
start migration only if ledger_version smaller
maciejdfinity Nov 21, 2024
0cac10f
proptest storable allowance
maciejdfinity Nov 21, 2024
6b07167
test minicbor serialization stable
maciejdfinity Nov 21, 2024
f4a8670
Merge branch 'master' into maciej-icrc-v3
maciejdfinity Nov 21, 2024
4ee6b80
build fix
maciejdfinity Nov 21, 2024
d498983
Merge branch 'maciej-icrc-v3' into maciej-icp-v3
maciejdfinity Nov 21, 2024
0b4ac9c
clear stable allowances in post_upgrade
maciejdfinity Nov 21, 2024
0fea156
Merge branch 'maciej-icrc-v3' into maciej-icp-v3
maciejdfinity Nov 21, 2024
d95f234
build fix
maciejdfinity Nov 26, 2024
f491986
build fix
maciejdfinity Nov 26, 2024
7daf32c
simple storable impl for account id, test
maciejdfinity Nov 26, 2024
7f56ab5
add low instruction limit wasm
maciejdfinity Nov 26, 2024
e372400
remove ledger_state from ledger, add is_ledger_ready endpoint
maciejdfinity Nov 26, 2024
cc2bb83
update migration code
maciejdfinity Nov 27, 2024
6b7a03f
refactor start migration logic
maciejdfinity Nov 27, 2024
eaeb034
update did file with is_ledger_ready
maciejdfinity Nov 27, 2024
4159aea
fix tests
maciejdfinity Nov 27, 2024
6e67fec
build fix
maciejdfinity Nov 27, 2024
b4b4008
build fix
maciejdfinity Nov 27, 2024
e47b434
fix tests
maciejdfinity Nov 27, 2024
27fdbd1
remove unused arrival fns
maciejdfinity Nov 27, 2024
1b44f1f
fix tests
maciejdfinity Nov 27, 2024
4a367af
fix tests
maciejdfinity Nov 28, 2024
e2f89a1
comment about new ledger version
maciejdfinity Nov 28, 2024
7489beb
better error messages
maciejdfinity Nov 28, 2024
7fe1e47
positive instruction limit for low limit upgrade
maciejdfinity Nov 28, 2024
efda1ae
comment
maciejdfinity Nov 28, 2024
92f8bec
typo
maciejdfinity Nov 28, 2024
8ef1538
typo
maciejdfinity Nov 28, 2024
7791b81
Merge branch 'master' into maciej-icp-v3
maciejdfinity Nov 28, 2024
8835ee2
merge error
maciejdfinity Nov 28, 2024
2ca367f
disallow migrating from V1
maciejdfinity Nov 28, 2024
613547f
merge error
maciejdfinity Nov 28, 2024
c1fe3db
merge error
maciejdfinity Nov 28, 2024
c21a538
typo
maciejdfinity Nov 28, 2024
b7c02aa
merge error
maciejdfinity Nov 28, 2024
cf9ddad
merge error
maciejdfinity Nov 28, 2024
c42fc0a
clippy
maciejdfinity Nov 28, 2024
25f1087
build fix
maciejdfinity Nov 28, 2024
ac44beb
counter
maciejdfinity Nov 28, 2024
63282a9
fix tests
maciejdfinity Nov 28, 2024
d9276bd
fix tests and logs
maciejdfinity Nov 28, 2024
aefa4e0
remove unused arg
maciejdfinity Nov 28, 2024
6d38f9f
replace fixed_tx test with multi step test
maciejdfinity Nov 28, 2024
be6545a
clippy
maciejdfinity Nov 28, 2024
8f2e2d4
icrc1 endpoints disabled test
maciejdfinity Nov 28, 2024
06acd50
more tests
maciejdfinity Nov 28, 2024
b7380a6
clippy
maciejdfinity Nov 28, 2024
58eee2f
proptest for allowance serialization
maciejdfinity Nov 28, 2024
2cebb08
Merge branch 'master' into maciej-icp-v3
maciejdfinity Nov 28, 2024
538dfca
remove unused dep
maciejdfinity Nov 28, 2024
c307548
test icp endpoints disabled
maciejdfinity Nov 29, 2024
2e98b9a
small refactor
maciejdfinity Nov 29, 2024
1e069ce
add to one disable endpoints test
maciejdfinity Nov 29, 2024
51d1a6f
print using dfn core
maciejdfinity Dec 3, 2024
16280b6
move proptest to dev deps
maciejdfinity Dec 3, 2024
45ef2aa
fix panic message
maciejdfinity Dec 3, 2024
bc58d30
Merge branch 'master' into maciej-icp-v3
maciejdfinity Dec 3, 2024
0f88412
Merge branch 'master' into maciej-icp-v3
maciejdfinity Dec 4, 2024
ffe19c8
Merge branch 'master' into maciej-icp-v3
maciejdfinity Dec 5, 2024
c4a4221
Merge branch 'master' into maciej-icp-v3
maciejdfinity Dec 9, 2024
5c32669
check endpoints disabled in main
maciejdfinity Dec 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 28 additions & 23 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 0 additions & 46 deletions rs/ledger_suite/common/ledger_canister_core/src/ledger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -332,52 +332,6 @@ where
.unwrap();
}

// We estimate that an approval takes up twice as much space as a balance:
// balance = account + num_tokens
// approval = 2 * account + num_tokens + timestamp
let max_number_of_approvals =
(effective_max_number_of_accounts - ledger.balances().store.len()) / 2;

if ledger.approvals().len() > max_number_of_approvals {
let num_approvals_to_trim = ledger.approvals().len() - max_number_of_approvals;
// There might be some more expired approvals to prune.
ledger.approvals_mut().prune(now, num_approvals_to_trim);

if ledger.approvals().len() > max_number_of_approvals {
let approvals_to_trim = ledger
.approvals()
.select_approvals_to_trim(ledger.approvals().len() - max_number_of_approvals);

for approval in approvals_to_trim {
let approve_tx = L::Transaction::approve(
approval.0,
approval.1,
L::Tokens::zero(),
Some(now),
Some(TRIMMED_MEMO),
);

approve_tx
.apply(ledger, now, L::Tokens::zero())
.expect("failed to reset approval to zero");

let parent_hash = ledger.blockchain().last_hash;
let fee_collector = ledger.fee_collector().cloned();

ledger
.blockchain_mut()
.add_block(L::Block::from_transaction(
parent_hash,
approve_tx,
now,
L::Tokens::zero(),
fee_collector,
))
.unwrap();
}
}
}

Ok((height, ledger.blockchain().last_hash.unwrap()))
}

Expand Down
Loading
Loading