Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ready attribute, when exchange is ready #2

Open
jpmckinney opened this issue Nov 19, 2021 · 4 comments
Open

Add ready attribute, when exchange is ready #2

jpmckinney opened this issue Nov 19, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@jpmckinney
Copy link
Member

jpmckinney commented Nov 19, 2021

Relevant code: https://github.com/pika/pika/blob/55557c9172edb04256a5e49b7faf13078d5a0f66/examples/blocking_consume_recover_multiple_hosts.py#L54-L63

@jpmckinney jpmckinney added the enhancement New feature or request label Nov 19, 2021
@jpmckinney
Copy link
Member Author

Witnessed exception classes

  • AMQPConnectorSocketConnectError: ConnectionRefusedError
  • AMQPConnectionError
  • StreamLostError: ConnectionResetError
  • ConnectionResetError

ConnectionWrongStateError also witnessed with "BlockingConnection.close(200, 'Normal shutdown') called on closed connection." – I assume this is yapw attempting a clean shutdown after one of the above unhandled exceptions.

@jpmckinney
Copy link
Member Author

jpmckinney commented Jan 25, 2022

This has been implemented in clients. Different clients might want to recover from different errors (and instantiate different clients), so it might be over-complicated to make this reusable in this library, as we'd need to pass the new function a "client factory".

open-contracting/data-registry@517bcf5#diff-5a647a7f71f26fcf662e3f8eab7eb913b41cea4219fb187871d12516fb2ad45b

@jpmckinney
Copy link
Member Author

jpmckinney commented Jan 25, 2024

Noting that we might need to set a "ready" state like we do in Kingfisher Collect and then only run functions when ready.

We can do this as a new function that calls exchange_ready after setting the ready state.

https://github.com/open-contracting/kingfisher-collect/blob/1db352034eb91e004aa50fe48fa8a14885f37dd0/kingfisher_scrapy/extensions/kingfisher_process_api2.py#L15-L25

It might be as simple as changing add_callback_threadsafe to check the ready state:

https://github.com/open-contracting/kingfisher-collect/blob/1db352034eb91e004aa50fe48fa8a14885f37dd0/kingfisher_scrapy/extensions/kingfisher_process_api2.py#L175-L180

@jpmckinney
Copy link
Member Author

Re-opening to upstream the Kingfisher Collect logic, essentially:

class Client(Async):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.ready = False

    def exchange_ready(self):
        self.ready = True

    def reset(self):
        super().reset()
        self.ready = False

Then document the ready attribute with an example of why/when to use it.

@jpmckinney jpmckinney reopened this Sep 27, 2024
@jpmckinney jpmckinney changed the title Add pattern to reconnect after connection errors Add ready attribute, when exchange is ready. Sep 27, 2024
@jpmckinney jpmckinney changed the title Add ready attribute, when exchange is ready. Add ready attribute, when exchange is ready Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant