points by lnkdinsuxs 4 days ago | 0 comments
post training is both the future and the secret sauce.
The next iteration of AI assisted devs will not have all the devs use the same harness and model like most do now.
Instead, devs will build their own post trained models that are experts of their own domain, that have their secret sauce..
I imagine any dev would want most of those models to be something they can self host because of multiple reasons - one of them being that their employers or customers might not agree to use a cloud hosted or managed model. Plus, costs. And control.
Post training is in the future because we haven't yet reached a shared agreement on what that first "good enough" model will be like - and also because the current AI scape is in a state of flux where post training will have a lot of churn.
Hopefully there will be more releases of open BASE LLMs, i.e. with the weights that are the results of pre-training, because those are much more suitable for being trained for whatever you want, than when you try to retrain fully-trained models and you have to undo whatever was done to them that is inappropriate for you.
The BASE LLMs have already done the part that requires a lot of resources, i.e collecting a huge data set, then running training on a big number of GPUs with a lot of memory and fast interconnections, and thus compressing by several orders of magnitude the training data set into the LLM weights.
Starting from there, training can be finalized in any desired direction with much less resources.
There is another on-going HN thread about such a BASE model that has just been released, baidu/ERNIE, which seems quite good. Several other companies, e.g. Google, NVIDIA, DeepSeek, Alibaba, have released in the past such BASE models suitable for training your own LLM.