Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twom database format and first implementation #5157

Open
wants to merge 39 commits into
base: master
Choose a base branch
from

Conversation

brong
Copy link
Member

@brong brong commented Dec 5, 2024

Throwing this open for wide reviews now! It's been rewritten a BUNCH of times, and optimised a fair bit - it's got a simpler internal structure and basically everything aligned with the right datastructures I think. It does MVCC reads nicely, can keep its transaction in foreach unless the callback writes, in which case it STILL does the right thing and re-locks and re-finds itself.

On disk format of records is:
image

lib/cyrusdb_twom.c Outdated Show resolved Hide resolved
@rjbs
Copy link
Collaborator

rjbs commented Dec 16, 2024

I've removed include-in-fm while tests are failing.

@brong brong force-pushed the twom branch 4 times, most recently from 7d3b303 to 5ffbf60 Compare January 5, 2025 11:37
@brong brong requested review from dilyanpalauzov, robmueller, rjbs, elliefm, wolfsage and rsto and removed request for dilyanpalauzov January 5, 2025 11:41
@brong brong force-pushed the twom branch 2 times, most recently from 981f5c7 to a0615b2 Compare January 5, 2025 12:14
@brong
Copy link
Member Author

brong commented Jan 5, 2025

Bah, CI failed because errno.h is required. I took it out because it's not needed in my machine and I was trying to trim back all the required headers

@brong brong requested a review from dilyanpalauzov January 5, 2025 12:33
Copy link
Contributor

@elliefm elliefm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't finished looking at this yet, but need to stop for lunch, so here's what I have so far.

imap/hammer_cyrusdb.c Outdated Show resolved Hide resolved
imap/hammer_cyrusdb.c Outdated Show resolved Hide resolved
lib/imapoptions Outdated Show resolved Hide resolved
lib/imapoptions Outdated Show resolved Hide resolved
lib/cyrusdb_twom.c Outdated Show resolved Hide resolved
lib/cyrusdb_twom.c Outdated Show resolved Hide resolved
@brong
Copy link
Member Author

brong commented Jan 25, 2025

Thanks @rsto I've added some more doc, made a few tiny fixes, and fixed up cyr_dbtool to do better error handling and patch readonly through.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants