when conducting the inference, do you set do_sample=True?

[renatz]

for the temperature non-zero, i think you set do_sample=True?
how about the top_p setting?

[grayground]

Thank you for your question.
For closed-source models, increasing the temperature parameter essentially equates to performing sampling (as evidenced by the results in the paper's section "The Impact of Sampling Temperature").
For open-source models, we did not specifically adjust the top_p parameter; in vicuna-7b-v1.5, the default value for the top_p parameter is 0.6.