Push tweets to a queue

[AetherUnbound]

Currently, tweets are coming from Twitter's "stream" API. Before I fixed an issue with trusted numbers, the tweets we wanted to send out via Signal were being slowed down significantly. This would backup the streaming API, and Twitter would cut off the but with the following error:

Full error

BOT FAILURE: ('Connection broken: IncompleteRead(4 bytes read)', IncompleteRead(4 bytes read))
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 555, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/usr/local/lib/python3.8/http/client.py", line 522, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 587, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/usr/local/lib/python3.8/http/client.py", line 557, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 436, in _error_catcher
    yield
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 518, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.8/http/client.py", line 458, in read
    n = self.readinto(b)
  File "/usr/local/lib/python3.8/http/client.py", line 492, in readinto
    return self._readinto_chunked(b)
http.client.IncompleteRead: IncompleteRead(4 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/signal_scanner_bot/transport.py", line 53, in twitter_to_signal
    await loop.run_in_executor(pool, _twitter_to_signal)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/signal_scanner_bot/transport.py", line 43, in _twitter_to_signal
    stream.filter(track=twitter.RECEIVE_HASHTAGS, stall_warnings=True)
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 474, in filter
    self._start(is_async)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(4 bytes read)', IncompleteRead(4 bytes read))

From what I can tell, this error is consistent with Twitter saying "you're not handling the tweets fast enough". The easiest way to prevent this backup and hopefully prevent the errors from happening is to have a queue between the tweet and the message sending interface. This relates to #22 as it may standardize the way both consumers handle messages.

[mpuckett159]

So I've been running a pretty stripped down version of the bot to push new Tweets from a single account to a Discord server, and I'm getting the same error message pretty frequently. I think this might be just a problem with the tweepy library? My Twitter to Discord bot is definitely not having issues waiting on the send to Discord. I'm going to be setting up a longer term test of the queue'ing code branch, but I'm starting to think it's just a problem with tweepy not handling connection resets gracefully.