Discrepancy between recommended temperatures and no default/greedy for `transformers`
#9
by
casinca
- opened
Hello @patrickvonplaten ,
(Posting here but same thing for Devstral small.)
In the blogpost, it is mentioned recommended temp=0.2, in the model card here it's 0.15. It's a detail but in any case it seems you recommend low temps.
for the transformers code snippet in the model card, it's going to use greedy decoding. It might lead to subpar perfs if people copy/paste/want to do local tests.
Additionally you could add a temperature parameter in generation_config.json so that people get your recommended sampling hparam by default with do_sample=True
I can open a PR if it's relevant