Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-1035 Fix toast slice fetching with offset #282

Merged
merged 1 commit into from
Sep 17, 2024

Conversation

artemgavrilov
Copy link
Collaborator

@artemgavrilov artemgavrilov commented Sep 17, 2024

https://perconadev.atlassian.net/browse/PG-1035

PG strings regression tests failed on section when it checks how substirng func works for toasted bytea data: https://github.com/postgres/postgres/blob/master/src/test/regress/sql/strings.sql#L575-L606

It seems that reason for that was that start_offset was calculated improperly. Looks like PG_TDE_DECRYPT_DATA should get start_offset alligned by chunk size, but when we copy data to result we can and should apply sliceoffcet.

Copy link

Performance test results:
Normal queries: 9126
TDE queries: 8809
Percentage: 96%

@artemgavrilov artemgavrilov marked this pull request as ready for review September 17, 2024 15:25
Copy link
Member

@dAdAbird dAdAbird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sliceoffset is redundant in the decryption offset. sliceoffset points to the offset from the start of TOAST value and (curchunk * TOAST_MAX_CHUNK_SIZE - sliceoffset) + chcpystrt will get the offset for each chunk from the start of fetched data. But for decryption, we need the offset relative to the beginning of TOAST data.
For example:
SELECT substring(t, 6660, 5984) FROM toasttest; will get 4 chunks and sliceoffset would be 6659. And offset with the sliceoffset correlation for these chunks would be 0, 1325, 3321, and 5317 - offset relative to the beginning of the requested data - which is what we need for memcpy data into the buffer. But for decryption, we need 6659, 7984, 9980, and 11976 offsets - relative to the beginning of the TOASTed data.

@artemgavrilov
Copy link
Collaborator Author

@dAdAbird Thanks for an explanation!

@artemgavrilov artemgavrilov merged commit 137a498 into main Sep 17, 2024
13 checks passed
@artemgavrilov artemgavrilov deleted the PG-1035-toasted-bytea-substring branch September 17, 2024 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants