About mm_projector loading issue

[rubylan]

Hi there,

Great work!

Just encountered a small issue when learning about the implementation.
When I check the repo (link here), 'mm_projector' is not included in the list of 'trainable_module', which means the module of mm_projector will not load its weights from the checkpoint (passing by --model_name_or_path) during stage-2/-3 or inference mode, right?

I read through the whole implementation carefully and found nowhere for the loading.
I am wondering if there is anything I've missed since it works well and should not have such a factual error.

Looking forward to your reply and thanks in advanced :))

Best,
Ruby

[Einstone-rose]

You can refer to Line 78-90 for details. Actually, mm_projector is the fully-connected network, and it is pretrained from scratch and as a member of module pretrain_mm_mlp_adapter.