More and more, the talk of using generative models (e.g. transformers) is coming up to replace existing models in all concepts. Anything from understanding data, image recognition, labeling, ranking, etc.

As someone who is not an MLE and really doesnt understand these models well, it seems like the transformer approach is a "one size fits all" solution that may not be optimal for all problem categories but is good enough that its not worth investing in specialized models to get incremental improvements.