The rate limits suck

[ioistired]

Conversion takes 9–20 minutes right now. This is unacceptable.

In the short term, concurrency levels need to be tuned in order to maximize rate limits. The concurrency levels would also ideally be based on a formula that factors in the per-account rate limits and the number of concurrent command invocations.

In the long term, discussion needs to happen with the signal team to work on some sort of private rate limit exemptions.

[ioistired]

Signal and Telegram both have clients first upload each individual image, then the pack metadata. Signal adds an additional request at the beginning to obtain credentials for uploading each image,

On the Signal side, on the level of one pack, the rate limit is 50 packs + an additional pack every 72 minutes (20 per day). The code that limits (or not?) individual image uploads is not public. In fact, it doesn't hit an API endpoint at all, but a CDN. So empirical testing is needed to determine the rate limits on CDN uploads, if any.

On the Telegram side, individual image uploads definitely do get "flood waited" so research is needed to see what those limits are. Pack creation is probably rate limited too, but image uploads are more likely the bottleneck here.

[derhagen]

I usually get an error message like this, independet of the sticker pack: An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 16121611191210955099

It comes up pretty much immediately, though. Is this due to some rate limit or an unrelated bug?

[ioistired]

@derhagen, it's probably rate limits, but I can't tell as I only keep the past 7 days of logs and I didn't get to your report until now.

[derhagen]

@iomintz Here we go again: An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 11512793617713579862

[ioistired]

Yeah that's rate limit exceeded. At the moment, whether Signal will allow you to run the command is kind of a lottery. While I work something out, you can alleviate the issue by running your own instance, and there's also a CLI planned so you don't have to set up signald to use it as well.

[derhagen]

I didn't get where the rate limit comes into play. Is it when handing the new sticker pack to signal and generating a signal.art link? If so, maybe, the pull request to signalstickers could be created anyway.

[ioistired]

Is it when handing the new sticker pack to signal and generating a signal.art link?

Yes

If so, maybe, the pull request to signalstickers could be created anyway.

No, as signalstickers.com requires a signal.art link.

[rugk]

An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 3708815344375093597

[Zerogoki00]

An internal error occurred while trying to run that command. Hey if you see the owner, give them this code okay? 3708815344375093597

I have the same problem