Distributed post training over the internet is much easier than distributed pretraining. What we need is just agent traces, from everyone's daily use. Hugging Face already supports uploading agent traces, see https://huggingface.co/docs/hub/en/agent-traces

We've been developing DataClaw to help everyone collect agent traces, see https://github.com/peteromallet/dataclaw . There are other similar efforts, such as in https://old.reddit.com/r/LocalLLaMA/comments/1u795pb/donate_...

The raw CoT is encrypted for closed source models, but there are plenty of ways to decrypt them. Even without CoT content, the tool calls and the length of CoT are already good signals for post training.

In China we use 'transition stations' to access GPT and Claude. They collect GBs of plain text agent traces every day, and the data eventually belong to the open source community. My estimation is that there are already 10-100 TB of agent traces freely on the internet for everyone to download.