Clarification on fine-tuning with MACE-mp-0 large vs. mace-mp-0b medium models

[AlghamdiNada]

Hello,

In the doc, it is emphasized that for multihead replay fine-tuning, one should use the MACE-mb-0b models. And here Ilyes mentioned that for large MACE-mp-0 models, it is recommended to use naive fine-tuning (multiheads_finetuning=False).

In my case, I have observed that the large MACE-mp-0 model provides better results compared to the medium mp-0b model for my specific problem. This makes me inclined to fine-tune the large mp-0 model.

Could someone clarify why the multihead replay fine-tuning is advised against for large mp-0 model? Is there a reason that makes the naive fine-tuning more suitable in this case?

Thanks a lot in advance!

[ilyes319]

Hey @AlghamdiNada, sorry for the delay.

The reason is that it is important to use actual E0s from DFT for multihead finetuning. The original MP-0 models were trained with E0s estimated as averages over the dataset and not by DFT single point. We changed that starting from MP-0b and the subsequent model. If you want to finetune a model now, I would recommend using our newest MPA-0 model that you can download here. It will be suitable for both multihead replay and normal finetuning and is quite accurate https://github.com/ACEsuit/mace-mp/releases/tag/mace_mpa_0.

For the multihead finetuning, I recommend you use the latest main branch, and try different --num_samples_pt, from 100 to 100 000. The higher the --num_samples_pt, the smaller the number epochs required as we repeat your own data (divide the number of epochs by num_samples_pt/100).