Skip to content

Commit

Permalink
Bitwise costing (PLT-8790) (#5733)
Browse files Browse the repository at this point in the history 
* Move conversion code into Plutus Core

* Documentation and notes on implementations

* Wrap implementations into builtins

* Properties as per CIP-0087

* CIP-0087 examples as tests

* Add new builtins to PlutusTx

* Document fromIntegral usage as a note

* Changelogs for CIP-0087 primitives

* Ensure conversions don't break on too-large arguments

* Ensure that conversions are available in V3

* Remove unnecessary pragmata on tests

* Fix overly-long test names, clarify test meaning in comments

* CIP link consistency

* Re-order integerToByteString arguments, avoid unnecessary padding

* Better documentation for implementations

* Correct properties for ByteStringToInteger

* Address feedback

* Integer/ByteString conversion costing experiments

* Turn off warning

* Turn off warning

* Initial costing for bitwise conversions

* Update print-cost-model; add some FIXMEs

* Update print-cost-model; add some FIXMEs

* Minor updates

* Fix typo

* Fix plugin; use byteStringToInteger in  bls12-318-costs

* Fix some tests

* Add generator for LiteralByteSize in plc evaluation tests

* Update parameter names in plutus-ledger-api

* Separate file for bitwise costing benchmarks

* More comprehensive exploratory costing benchmarks

* More comprehensive exploratory costing benchmarks

* Wrong test input sizes

* Correct costing benchmarks input widths

* One more benchmark

* All 0xFF benchmarks

* Wrong number

* Smaller bytetring inputs

* Try to keep CI happy

* Bigger benchmarks

* Another benchmarking run

* Fix modelling code for integer/bytestring conversions

* Remove experimental benchmarks; tidy up R code

* Tidying up; fix budget benchmarks for byteStringToInteger

* Move bitwise Convert module

* Move bitwise Convert module

* Update cost model

* Update bls12-381 cost test results

* Update comment

* Add changelog entries

* Remove accidentally-committed file

* Cost model tests for bitwise conversions

* Fix failing test

* Fix another failing test

* Address some small PR comments

* Address some small PR comments

* Initial conformance tests for bitwise conversion builtins

* Move size limit check back to Convert.hs and add some tests

* Move size limit check back to Convert.hs and add some tests

* Workaround for integerLog2 missing in GHC 8.10

* Workaround for integerLog2 missing in GHC 8.10

* Workaround for integerLog2 missing in GHC 8.10

* Add some more test cases

* More test cases

* More test cases

* More tests

* More test cases

* Formatting

* Update comment

* Fix PLC version number in bitwise conformance tests

* Update golden tests for new maximum width

* Address PR comments

* Test output mysteriously rearranged again

---------

Co-authored-by: Koz Ross <[email protected]>
  • Loading branch information
@kwxm @kozross
kwxm and kozross authored Jan 26, 2024
1 parent 0c23785 commit e3de827
Show file tree
Hide file tree
Showing 120 changed files with 1,527 additions and 434 deletions.
31 changes: 13 additions & 18 deletions plutus-benchmark/bls12-381-costs/src/PlutusBenchmark/BLS12_381/Scripts.hs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ import Data.ByteString (ByteString)
import Data.ByteString qualified as BS
import Data.ByteString.Char8 qualified as C8
import Data.Semigroup ((<>))
import GHC.ByteOrder (ByteOrder (LittleEndian))
import Hedgehog.Internal.Gen qualified as G
import Hedgehog.Internal.Range qualified as R
import System.IO.Unsafe (unsafePerformIO)
Expand Down Expand Up @@ -79,16 +80,10 @@ blsSigBls12381G2XmdSha256SswuRoNul = toBuiltin $ C8.pack "BLS_SIG_BLS12381G2_XMD
byteString16Null :: BuiltinByteString
byteString16Null = bytesFromHex "00000000000000000000000000000000"

-- workaround the lack of ByteString to Integer interpretation
-- replace with builtin when available, see https://github.com/IntersectMBO/plutus/pull/4733
byteStringToInteger :: BuiltinByteString -> Integer
byteStringToInteger b =
go 0
where len = Tx.lengthOfByteString b
go i =
if i >= len
then 0
else (Tx.indexByteString b i) + 256 * (go (i + 1))
-- Little-endian bytestring to integer conversion
{-# INLINABLE byteStringToIntegerLE #-}
byteStringToIntegerLE :: BuiltinByteString -> Integer
byteStringToIntegerLE = Tx.byteStringToInteger LittleEndian

---------------- Examples ----------------

Expand Down Expand Up @@ -386,9 +381,9 @@ vrfBlsScript message pubKey (VrfProofWithOutput beta (VrfProof gamma c s)) =
-- do the following calculation
pubKey' = Tx.bls12_381_G2_uncompress pubKey
g2generator = Tx.bls12_381_G2_uncompress bls12_381_G2_compressed_generator
u = Tx.bls12_381_G2_add (Tx.bls12_381_G2_scalarMul (byteStringToInteger c) pubKey') (Tx.bls12_381_G2_scalarMul s g2generator)
u = Tx.bls12_381_G2_add (Tx.bls12_381_G2_scalarMul (byteStringToIntegerLE c) pubKey') (Tx.bls12_381_G2_scalarMul s g2generator)
h = Tx.bls12_381_G2_hashToGroup message emptyByteString
v = Tx.bls12_381_G2_add (Tx.bls12_381_G2_scalarMul (byteStringToInteger c) (Tx.bls12_381_G2_uncompress gamma)) (Tx.bls12_381_G2_scalarMul s h)
v = Tx.bls12_381_G2_add (Tx.bls12_381_G2_scalarMul (byteStringToIntegerLE c) (Tx.bls12_381_G2_uncompress gamma)) (Tx.bls12_381_G2_scalarMul s h)
-- and check

in c == (sha2_256 . mconcat $ Tx.bls12_381_G2_compress <$> [g2generator, h, pubKey', Tx.bls12_381_G2_uncompress gamma, u, v])
Expand Down Expand Up @@ -422,7 +417,7 @@ generateVrfProof privKey message =
<$> [g2generator, h, pub, gamma, Tx.bls12_381_G2_scalarMul k g2generator, Tx.bls12_381_G2_scalarMul k h]

-- define the third and last element of a proof of correct VRF
s = (k - (byteStringToInteger c) * privKey) `modulo` 52435875175126190479447740508185965837690552500527637822603658699938581184513
s = (k - (byteStringToIntegerLE c) * privKey) `modulo` 52435875175126190479447740508185965837690552500527637822603658699938581184513

-- cofactor of G2
f = 305502333931268344200999753193121504214466019254188142667664032982267604182971884026507427359259977847832272839041692990889188039904403802465579155252111 :: Integer
Expand Down Expand Up @@ -704,7 +699,7 @@ aggregateMultiKeyG2Script message pubKeys aggregateSignature bs16Null dst =
hashedMsg = Tx.bls12_381_G1_hashToGroup message dst
pksDeser = Tx.map Tx.bls12_381_G2_uncompress pubKeys
-- scalar calcuates to (142819114285630344964654001480828217341 :: Integer)
dsScalar = byteStringToInteger (Tx.sliceByteString 0 16
dsScalar = byteStringToIntegerLE (Tx.sliceByteString 0 16
(Tx.sha2_256 (foldl1 Tx.appendByteString pubKeys)) `Tx.appendByteString` bs16Null)
aggrSigDeser = Tx.bls12_381_G1_uncompress aggregateSignature
aggrPk = calcAggregatedPubkeys dsScalar pksDeser
Expand Down Expand Up @@ -810,11 +805,11 @@ schnorrG1VerifyScript ::
schnorrG1VerifyScript message pubKey signature bs16Null =
let a = Tx.fst signature
r = Tx.snd signature
c = byteStringToInteger (Tx.sliceByteString 0 16
c = byteStringToIntegerLE (Tx.sliceByteString 0 16
(Tx.sha2_256 (a `Tx.appendByteString` pubKey `Tx.appendByteString` message)) `Tx.appendByteString` bs16Null)
pkDeser = Tx.bls12_381_G1_uncompress pubKey
aDeser = Tx.bls12_381_G1_uncompress a
rDeser = byteStringToInteger r
rDeser = byteStringToIntegerLE r
g1generator = Tx.bls12_381_G1_uncompress Tx.bls12_381_G1_compressed_generator
in (rDeser `Tx.bls12_381_G1_scalarMul` g1generator) ==
(aDeser `Tx.bls12_381_G1_add` (c `Tx.bls12_381_G1_scalarMul` pkDeser))
Expand Down Expand Up @@ -876,11 +871,11 @@ schnorrG2VerifyScript ::
schnorrG2VerifyScript message pubKey signature bs16Null =
let a = Tx.fst signature
r = Tx.snd signature
c = byteStringToInteger (Tx.sliceByteString 0 16
c = byteStringToIntegerLE (Tx.sliceByteString 0 16
(Tx.sha2_256 (a `Tx.appendByteString` pubKey `Tx.appendByteString` message)) `Tx.appendByteString` bs16Null)
pkDeser = Tx.bls12_381_G2_uncompress pubKey
aDeser = Tx.bls12_381_G2_uncompress a
rDeser = byteStringToInteger r
rDeser = byteStringToIntegerLE r
g2generator = Tx.bls12_381_G2_uncompress Tx.bls12_381_G2_compressed_generator
in rDeser `Tx.bls12_381_G2_scalarMul` g2generator ==
(aDeser `Tx.bls12_381_G2_add` (c `Tx.bls12_381_G2_scalarMul` pkDeser))
Expand Down
138 changes: 69 additions & 69 deletions plutus-benchmark/bls12-381-costs/test/9.6/bls12-381-costs.golden
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,143 +3,143 @@ Hash n bytestrings onto G1 and add points
n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 77 (0.5%) 100 (0.0%) 100 (0.0%)
10 187 (1.1%) 681707800 (6.8%) 39342 (0.3%)
20 297 (1.8%) 1363794920 (13.6%) 75702 (0.5%)
30 407 (2.5%) 2045882040 (20.5%) 112062 (0.8%)
40 517 (3.2%) 2727969160 (27.3%) 148422 (1.1%)
50 627 (3.8%) 3410056280 (34.1%) 184782 (1.3%)
60 737 (4.5%) 4092143400 (40.9%) 221142 (1.6%)
70 847 (5.2%) 4774230520 (47.7%) 257502 (1.8%)
80 957 (5.8%) 5456317640 (54.6%) 293862 (2.1%)
90 1067 (6.5%) 6138404760 (61.4%) 330222 (2.4%)
100 1177 (7.2%) 6820491880 (68.2%) 366582 (2.6%)
110 1287 (7.9%) 7502579000 (75.0%) 402942 (2.9%)
120 1397 (8.5%) 8184666120 (81.8%) 439302 (3.1%)
130 1507 (9.2%) 8866753240 (88.7%) 475662 (3.4%)
140 1617 (9.9%) 9548840360 (95.5%) 512022 (3.7%)
150 1727 (10.5%) 10230927480 (102.3%) 548382 (3.9%)
10 187 (1.1%) 676580182 (6.8%) 39342 (0.3%)
20 297 (1.8%) 1353326072 (13.5%) 75702 (0.5%)
30 407 (2.5%) 2030071962 (20.3%) 112062 (0.8%)
40 517 (3.2%) 2706817852 (27.1%) 148422 (1.1%)
50 627 (3.8%) 3383563742 (33.8%) 184782 (1.3%)
60 737 (4.5%) 4060309632 (40.6%) 221142 (1.6%)
70 847 (5.2%) 4737055522 (47.4%) 257502 (1.8%)
80 957 (5.8%) 5413801412 (54.1%) 293862 (2.1%)
90 1067 (6.5%) 6090547302 (60.9%) 330222 (2.4%)
100 1177 (7.2%) 6767293192 (67.7%) 366582 (2.6%)
110 1287 (7.9%) 7444039082 (74.4%) 402942 (2.9%)
120 1397 (8.5%) 8120784972 (81.2%) 439302 (3.1%)
130 1507 (9.2%) 8797530862 (88.0%) 475662 (3.4%)
140 1617 (9.9%) 9474276752 (94.7%) 512022 (3.7%)
150 1727 (10.5%) 10151022642 (101.5%) 548382 (3.9%)


Hash n bytestrings onto G2 and add points

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 77 (0.5%) 100 (0.0%) 100 (0.0%)
10 187 (1.1%) 2075992430 (20.8%) 39684 (0.3%)
20 297 (1.8%) 4153677170 (41.5%) 76404 (0.5%)
30 407 (2.5%) 6231361910 (62.3%) 113124 (0.8%)
40 517 (3.2%) 8309046650 (83.1%) 149844 (1.1%)
50 627 (3.8%) 10386731390 (103.9%) 186564 (1.3%)
60 737 (4.5%) 12464416130 (124.6%) 223284 (1.6%)
70 847 (5.2%) 14542100870 (145.4%) 260004 (1.9%)
80 957 (5.8%) 16619785610 (166.2%) 296724 (2.1%)
90 1067 (6.5%) 18697470350 (187.0%) 333444 (2.4%)
100 1177 (7.2%) 20775155090 (207.8%) 370164 (2.6%)
110 1287 (7.9%) 22852839830 (228.5%) 406884 (2.9%)
120 1397 (8.5%) 24930524570 (249.3%) 443604 (3.2%)
130 1507 (9.2%) 27008209310 (270.1%) 480324 (3.4%)
140 1617 (9.9%) 29085894050 (290.9%) 517044 (3.7%)
150 1727 (10.5%) 31163578790 (311.6%) 553764 (4.0%)
10 187 (1.1%) 2070864812 (20.7%) 39684 (0.3%)
20 297 (1.8%) 4143208322 (41.4%) 76404 (0.5%)
30 407 (2.5%) 6215551832 (62.2%) 113124 (0.8%)
40 517 (3.2%) 8287895342 (82.9%) 149844 (1.1%)
50 627 (3.8%) 10360238852 (103.6%) 186564 (1.3%)
60 737 (4.5%) 12432582362 (124.3%) 223284 (1.6%)
70 847 (5.2%) 14504925872 (145.0%) 260004 (1.9%)
80 957 (5.8%) 16577269382 (165.8%) 296724 (2.1%)
90 1067 (6.5%) 18649612892 (186.5%) 333444 (2.4%)
100 1177 (7.2%) 20721956402 (207.2%) 370164 (2.6%)
110 1287 (7.9%) 22794299912 (227.9%) 406884 (2.9%)
120 1397 (8.5%) 24866643422 (248.7%) 443604 (3.2%)
130 1507 (9.2%) 26938986932 (269.4%) 480324 (3.4%)
140 1617 (9.9%) 29011330442 (290.1%) 517044 (3.7%)
150 1727 (10.5%) 31083673952 (310.8%) 553764 (4.0%)


Uncompress n G1 points and add the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 69 (0.4%) 100 (0.0%) 100 (0.0%)
10 619 (3.8%) 183823250 (1.8%) 37042 (0.3%)
20 1169 (7.1%) 368094820 (3.7%) 71402 (0.5%)
30 1719 (10.5%) 552366390 (5.5%) 105762 (0.8%)
40 2269 (13.8%) 736637960 (7.4%) 140122 (1.0%)
50 2819 (17.2%) 920909530 (9.2%) 174482 (1.2%)
60 3369 (20.6%) 1105181100 (11.1%) 208842 (1.5%)
70 3919 (23.9%) 1289452670 (12.9%) 243202 (1.7%)
80 4469 (27.3%) 1473724240 (14.7%) 277562 (2.0%)
90 5019 (30.6%) 1657995810 (16.6%) 311922 (2.2%)
100 5569 (34.0%) 1842267380 (18.4%) 346282 (2.5%)
110 6119 (37.3%) 2026538950 (20.3%) 380642 (2.7%)
120 6669 (40.7%) 2210810520 (22.1%) 415002 (3.0%)
130 7219 (44.1%) 2395082090 (24.0%) 449362 (3.2%)
140 7769 (47.4%) 2579353660 (25.8%) 483722 (3.5%)
150 8319 (50.8%) 2763625230 (27.6%) 518082 (3.7%)
10 619 (3.8%) 180114262 (1.8%) 37042 (0.3%)
20 1169 (7.1%) 360463232 (3.6%) 71402 (0.5%)
30 1719 (10.5%) 540812202 (5.4%) 105762 (0.8%)
40 2269 (13.8%) 721161172 (7.2%) 140122 (1.0%)
50 2819 (17.2%) 901510142 (9.0%) 174482 (1.2%)
60 3369 (20.6%) 1081859112 (10.8%) 208842 (1.5%)
70 3919 (23.9%) 1262208082 (12.6%) 243202 (1.7%)
80 4469 (27.3%) 1442557052 (14.4%) 277562 (2.0%)
90 5019 (30.6%) 1622906022 (16.2%) 311922 (2.2%)
100 5569 (34.0%) 1803254992 (18.0%) 346282 (2.5%)
110 6119 (37.3%) 1983603962 (19.8%) 380642 (2.7%)
120 6669 (40.7%) 2163952932 (21.6%) 415002 (3.0%)
130 7219 (44.1%) 2344301902 (23.4%) 449362 (3.2%)
140 7769 (47.4%) 2524650872 (25.2%) 483722 (3.5%)
150 8319 (50.8%) 2704999842 (27.0%) 518082 (3.7%)


Uncompress n G2 points and add the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
0 69 (0.4%) 100 (0.0%) 100 (0.0%)
10 1099 (6.7%) 361567910 (3.6%) 37384 (0.3%)
20 2129 (13.0%) 724897130 (7.2%) 72104 (0.5%)
30 3159 (19.3%) 1088226350 (10.9%) 106824 (0.8%)
40 4189 (25.6%) 1451555570 (14.5%) 141544 (1.0%)
50 5219 (31.9%) 1814884790 (18.1%) 176264 (1.3%)
60 6249 (38.1%) 2178214010 (21.8%) 210984 (1.5%)
70 7279 (44.4%) 2541543230 (25.4%) 245704 (1.8%)
80 8309 (50.7%) 2904872450 (29.0%) 280424 (2.0%)
90 9339 (57.0%) 3268201670 (32.7%) 315144 (2.3%)
100 10369 (63.3%) 3631530890 (36.3%) 349864 (2.5%)
110 11399 (69.6%) 3994860110 (39.9%) 384584 (2.7%)
120 12429 (75.9%) 4358189330 (43.6%) 419304 (3.0%)
130 13459 (82.1%) 4721518550 (47.2%) 454024 (3.2%)
140 14489 (88.4%) 5084847770 (50.8%) 488744 (3.5%)
150 15519 (94.7%) 5448176990 (54.5%) 523464 (3.7%)
10 1099 (6.7%) 357858922 (3.6%) 37384 (0.3%)
20 2129 (13.0%) 717265542 (7.2%) 72104 (0.5%)
30 3159 (19.3%) 1076672162 (10.8%) 106824 (0.8%)
40 4189 (25.6%) 1436078782 (14.4%) 141544 (1.0%)
50 5219 (31.9%) 1795485402 (18.0%) 176264 (1.3%)
60 6249 (38.1%) 2154892022 (21.5%) 210984 (1.5%)
70 7279 (44.4%) 2514298642 (25.1%) 245704 (1.8%)
80 8309 (50.7%) 2873705262 (28.7%) 280424 (2.0%)
90 9339 (57.0%) 3233111882 (32.3%) 315144 (2.3%)
100 10369 (63.3%) 3592518502 (35.9%) 349864 (2.5%)
110 11399 (69.6%) 3951925122 (39.5%) 384584 (2.7%)
120 12429 (75.9%) 4311331742 (43.1%) 419304 (3.0%)
130 13459 (82.1%) 4670738362 (46.7%) 454024 (3.2%)
140 14489 (88.4%) 5030144982 (50.3%) 488744 (3.5%)
150 15519 (94.7%) 5389551602 (53.9%) 523464 (3.7%)


Apply pairing to two pairs of points in G1 x G2 and run finalVerify on the results

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 342 (2.1%) 1293505872 (12.9%) 4654 (0.0%)
- 342 (2.1%) 1292150444 (12.9%) 4654 (0.0%)


Groth16 verification example

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 779 (4.8%) 2316309514 (23.2%) 10904 (0.1%)
- 779 (4.8%) 2312779174 (23.1%) 10904 (0.1%)

VRF example

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 753 (4.6%) 1364980692 (13.6%) 184634 (1.3%)
- 714 (4.4%) 1303368563 (13.0%) 49449 (0.4%)

G1 Verify

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 332 (2.0%) 1465218237 (14.7%) 5754 (0.0%)
- 332 (2.0%) 1463720946 (14.6%) 5754 (0.0%)

G2 Verify

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 380 (2.3%) 1343563718 (13.4%) 5754 (0.0%)
- 380 (2.3%) 1342066427 (13.4%) 5754 (0.0%)

Aggregate Single Key

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 777 (4.7%) 3343214820 (33.4%) 71202 (0.5%)
- 777 (4.7%) 3336910422 (33.4%) 71202 (0.5%)

Aggregate Multi Key

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 1743 (10.6%) 3741726913 (37.4%) 566293 (4.0%)
- 1701 (10.4%) 3676960887 (36.8%) 430886 (3.1%)

Schnorr Signature G1

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 418 (2.6%) 361167811 (3.6%) 264488 (1.9%)
- 370 (2.3%) 248136411 (2.5%) 13796 (0.1%)

Schnorr Signature G2

n Script size CPU usage Memory usage
----------------------------------------------------------------------
- 562 (3.4%) 606243489 (6.1%) 264656 (1.9%)
- 514 (3.1%) 493212089 (4.9%) 13964 (0.1%)

Groth16Verify succeeded
Simple Verify succeeded
Expand Down
Loading

0 comments on commit e3de827

Please sign in to comment.