But i also get the immediate "but how is anyone supposed to run this locally" feeling

Maybe we need a new network. Like, Nextdoor + counterstrike pub servers, but for local LLM. We can call it semilocal

:)