Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove blocking/synchronous code inside async blocks #4

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

llebout
Copy link

@llebout llebout commented Jan 18, 2020

This is critical as it could hang the entire runtime and/or affect performance very negatively.
It is pointless to use async blocks if it's to use synchronous code inside them, as the major benefit is performance, otherwise you're better off with using only synchronous code.

I understand that the underlying database adapter is synchronous, therefore, that one should be made asynchronous first.

async-std claims to offer a mechanism to dynamically detect and move blocking code off the main asynchronous runtime threads, this helps, but it's still bad for performance.

  • Use asynchronous Mutex
  • Make database adapter asynchronous
  • Make parallel message verification asynchronous

Copy link
Author

@llebout llebout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's all places where blocking code is called.

@@ -329,17 +329,22 @@ fn main() -> Result<(), Error> {
eprintln!("got {} new messages from server", packets.len());

let previous: Option<Vec<u8>> =
// This could block and get the entire async runtime to hang.
db.get_entry_by_seq(&author, latest_seq).unwrap();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking code here.

db.get_entry_by_seq(&author, latest_seq).unwrap();

// Later, we should add stuff to store into about broken feeds in the db.
// We should store why they broke and even store the offending message.
// Then we can do a avoid trying to replicate broken feeds over and over.

// This could block and get the entire async runtime to hang.
par_validate_message_hash_chain_of_feed(&packets, previous.as_ref()).unwrap();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here.

par_validate_message_hash_chain_of_feed(&packets, previous.as_ref()).unwrap();
eprintln!("validated messages");

// This could block and get the entire async runtime to hang.
par_verify_messages(&packets, None).unwrap();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here.

par_verify_messages(&packets, None).unwrap();
eprintln!("verified messages");

// This could block and get the entire async runtime to hang.
db.append_batch(&author, &packets).unwrap();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here.

rustle/src/rpc.rs Outdated Show resolved Hide resolved
let db = self.db.lock().unwrap();
let db = self.db.lock().await;

// This could block and get the entire async runtime to hang.
db.get_entries_newer_than_sequence(
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant