Temperature, top-up, top-k, min-p all control which token the model predicts next and how likely it is to select one token over the other.
You might understand this as "The capital of France is..." and the model isn't always going to select "Paris". Sometimes it will start a descriptive sentence or even get the answer wrong.
That selection of the next token is what these settings control, and lots of sub-optimal selections compound over time to produce a junk response.
Top-K: example setting 20. Select only from the 20 most likely tokens.
Top-P: example setting 0.9. Select tokens whose probably accumulates to this number. So say you have tokens with 0.7 then 0.2 then 0.1, the last will not be selected because the first two tokens already accumulated to >=0.9.
Min-P: example setting 0.05. Don't select tokens less probable than this value. So a token with 0.1 would be considered, a token with 0.01 would not.
The purpose of all of these is to exclude very unlikely next tokens.
Min-p is specifically "Don't select tokens less probable than a multiple of the top token's probability" with min_p of 0.1 multiplied by an example top probability of 0.3 being 0.03 as the truncation at that time step.
I've run the inference to get the answers I linked to. If someone else does the same thing, that involves extra energy. If I read your conversation instead of generating my own, then that's one less tree that has to be chopped down.
Until we go advanced geothermal or we crack fusion, energy is dirty. Read my inference or link me to yours so we don't boil the planet.
The Qwen team published the same sampler settings for 3.8 and presumably they used those while testing on benchmark. Do you believe they could have achieved higher result with top-n-sigma?
Diverging from the sampler used in RL training is not good for long multi-turn results-- it's a great way to knock models into reasoning loops that wouldn't otherwise.
If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later.