Video performance

Generating videos is very heavy and requires a lot of CPU and memory.

This page has some tips and tricks to improve video performance, most of them should be obvious to anyone that already worked with images/video generation, but still has some nifty tricks for newbies.

The tips are sorted from "easiest" to "hardest"

Changing ffmpeg's preset can improve your video processing speed, at the cost of disk space

The default preset medium, while generates small files, uses a lot of time and CPU.

While ultrafast gives you the best performance but the file sizes are too big to be sent via Discord (8MB+), so it is recommended to use superfast because that still gives very good performance while keeping the file size small.

If superfast is still too big, check other presets! https://trac.ffmpeg.org/wiki/Encode/H.264#Preset

ffmpeg -preset superfast

Frames should be stored in jpeg, unless if you have transparency

When loading frames, the performance is as following: bmp > jpg > png

• bmp is way too big to store frames, so don't use that.
• jpg file sizes are small, but remember that jpeg is lossy!
• png preserves quality, but it takes too long to load and also uses a lot of storage (less than bmp)

So, if your frames doesn't has transparency, use jpg or, if only some frames has transparency, do a mix of jpg and png

Loading jpeg optimized images with jpegoptim takes longer

I'm not sure why that happens but I guess it is related to the jpeg decompression algorithm.

ImageIO.write(...) bmp performance is the best of all (but also uses a lot of memory)

When doing frame edits, the performance is as following: bmp > jpg > png

But keep in mind bmp uses A LOT of memory! So do not try to do all the frames in parallel at once, this will cause a OutOfMemoryException

If a frame doesn't has any edits, do not load it via ImageIO

Loading images via ImageIO consumes a lot of memory and time so, if you have frames that does not have any edits on it, loading it via ImageIO wastes time.

The best way is to load the file contents (file.readAllBytes()) and send that directly to ffmpeg, keep in mind that if you are writing bmp for your edited frames but your stored frames are in jpeg/png/etc, ffmpeg will fail to encode the video.

To workaround that you can store the frames in the same format as your edited frames (keep in mind that bmp frames are HUGE) or keep them loaded into memory by loading with ImageIO and then writing it in a format compatible with your edited frames.

Reduce framerate if there is a lot of users using the generator

This is pretty easy, just change ffmpeg's framerate parameter to the half of the current framerate and make your code only generate half the frames (so it will take half the time to finish!), let's suppose your video is 30 FPS and has 991 frames example:

(0..990) -> (0..990 step 2) will generate half the frames (991 = ~495), skipping every other frame.

ffmpeg -framerate 15 will reduce the framerate of the video to 15 FPS.

Optimize the image generation loop

This is obvious, but it is also the hardest to do if you don't have low hanging fruit to fix. Every single ms matters when you are generating 100+ frames.

Avoid doing image resize inside of the generation loop, resize your images before editing them!