[Bug]: Activating preanalysis in av1_amf in ffmpeg on linux fails

[robertosw]

Describe the bug
This command results in the error below:

ffmpeg -hide_banner \
  -i movie.mkv \
  -c:v av1_amf \
  -quality quality \
  -rc vbr_peak \
  -align 64x16 \
  -preanalysis 1 \
  -t 600s \
  out.mp4

Input #0, matroska,webm, from 'movie.mkv':
  Metadata:
    ENCODER         : Lavf59.33.100
  Duration: 01:42:22.93, start: -0.043000, bitrate: 6381 kb/s
  Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default) (original)
      Metadata:
        ENCODER         : Lavc59.48.100 libx265
        DURATION        : 01:42:22.929000000
  Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp (default) (original)
      Metadata:
        ENCODER         : Lavc59.48.100 libfdk_aac
        DURATION        : 01:42:22.891000000
  Stream #0:2(eng): Subtitle: subrip (srt) (default) (original)
      Metadata:
        DURATION        : 01:42:15.671000000
  Stream #0:3(eng): Subtitle: subrip (srt) (original) (hearing impaired)
      Metadata:
        title           : SDH
        DURATION        : 01:42:17.465000000
  Stream #0:4(spa): Subtitle: subrip (srt) (dub)
      Metadata:
        title           : Latin American
        DURATION        : 01:42:15.671000000
File 'cropped_quality_vbr-peak_8M.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> av1 (av1_amf))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[av1_amf @ 0x37b26580] rate control mode is PEAK_CONSTRAINED_VBR but rc_max_rate is not set
[av1_amf @ 0x37b26580] encoder->Init() failed with error 4
[vost#0:0/av1_amf @ 0x37b0fe40] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x37b10d00] Error sending frames to consumers: Internal bug, should not have happened
[vf#0:0 @ 0x37b10d00] Task finished with error code: -558323010 (Internal bug, should not have happened)
[vf#0:0 @ 0x37b10d00] Terminating thread with return code -558323010 (Internal bug, should not have happened)
[vost#0:0/av1_amf @ 0x37b0fe40] Could not open encoder before EOF
[vost#0:0/av1_amf @ 0x37b0fe40] Task finished with error code: -22 (Invalid argument)
[vost#0:0/av1_amf @ 0x37b0fe40] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0x37ac4180] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A dup=1 drop=0 speed=N/A    
[aac @ 0x38928580] Qavg: 36169.434
Conversion failed!

However once -preanalysis 1 is removed from the command, everything works.

Setup:

• OS: NixOS 24.11 - kernel 6.10.9
• Driver Version:
  • AMD open source driver 2024.Q3.2 (LLPC)
  • OpenCL 2.1 AMD-APP (3602.0) - FULL_PROFILE
• GPU: RX 7800 XT
• Which component has the issue: AV1 Encoder

Expected behavior
I'd expect ffmpeg to continue working.

Please let me know if this is the wrong place for this issue or how I could provide more context

[robertosw]

After reading some of the other issues I found this comment: #468 (comment).
Could the origin of this problem be that preanalysis is not allowed for -rc vbr_peak ?
I've not been using the other rate control modes, because they dont work, even with -preanalysis true

[robertosw]

Sorry for the trouble, I just didn't find this any earlier. Preanalysis only works with pixel format NV12
With this pixel format the other rate controls also work. Is this intended behavior?

Additionally, I noticed that

1. Using QVBR / HQVBR / HQCBR without -preanalysis true wont start the encoding process.
2. The encoded video repeatably jumps back and forth a few frames at playback when using one of these three rate controls. This looks like the order of I, P, B Frames got messed up. This happens when playing with ffplay, firefox or vlc. Should I open another issue for this?

The jumpy videos were encoded with this command and no errors were displayed

ffmpeg -hide_banner \
    -i movie.mkv \
    -c:v av1_amf \
    -pix_fmt nv12 \
    -quality speed \
    -rc qvbr \
    -preanalysis true \
    -align 64x16 \
    -b:v 8M \
    -t 120s \
    test.mp4