Thank you for bringing it up. I added BERT intentionally to draw attention on this fact that I'm unable to convey properly yet: I was okay with the use of previous generation of models, so it's not that I'm against LLMs since BERT-like architectures were indeed used.

What worries me about the current generation of models is privacy. In another comment, I tried to draw the line where models start to worry me regarding it. Discussing it with whimsicalism, we noticed that it's not the generative aspect either; model size is probably the main factor. For instance, a larger encoder could make its embedding transform almost lossless and thus more prone to data leakage again, although a decoder-like makes it more susceptible.