Which parameter to vary depends on the result of the test, but the MaxxBench route of varying the task spec seems like a good default, though not always the most informative.
In one of my private benchmarks I realised many of the less capable models didn't fail because they submitted the wrong answer, but because they exceeded their budget for number of tool calls they were allowed. The more capable models never got close to the tool call limit because they tended toward combining tool calls (e.g. chaining multiple shell commands in one Bash tool call).
Was my benchmark actually measuring ability to combine tool calls rather than reasoning ability on the problem at hand? Well, easy enough to find out: re-run under a different tool call budget and see how much the scores change.
(Next problem: is ability to combine tool calls a useful thing to measure, i.e. should it confer an advantage in the benchmark? It could be if it is a sign of planning multiple steps ahead.)