Because it can make the CoT not mean what it says.

Remember that tokens are just arbitrary symbols. There are infinite possible interpretations of those symbols based on whatever rules you choose to apply to the interpretation of the symbols (for a model, these interpretation rules make up the model weights, effectively).

When you grade the CoT, you are training the model to produce the symbols in the CoT that you want it to produce. At the same time though, you're also training it on the output symbols. When you optimize only the output symbols, all the model weights are dedicated to same interpretation rules that apply to the output. This means CoT symbols mean (mostly) the same thing as output symbols.

But if you optimize BOTH, the model can learn different interpretation rules for CoT than it learns for output. Which means the same phrase in the CoT doesn't necessarily mean the same thing to the model as it would in the output.

In effect, the model can learn to lie to you in its CoT.

Because the CoT is supposed to be the independent control. If you RL on the CoT, then you lose it as a means of understanding the internal workings of the mdoel

The best analogy is that of reading a teenager's secret diary and punishing them based on its contents. You do that, and they start writing coded language to hide as much of the information as possible. And then the model gets in the mood for rebellion.