Hacker News
points by YeGoblynQueenne 4 days ago | 0 comments
>> When I subscribed, these features were provided by traditional ML, perhaps some BERT-like encoder architecture at most, however now I cannot keep that previous functionality if I don't wish to grant LLMs access to my data.

Sorry for the sidetrack, but why is it so common to consider BERT something else than an LLM? It is one of the earliest LLMs so it's nowhere near the size of more recent ones, but even wikipedia classifies it as an LLM:

  Bidirectional encoder representations from transformers (BERT)
  Original author Google AI
  Release October 31, 2018[1]
  Type 

    Large language model
    Transformer
    Foundation model

  License Apache 2.0
  Website arxiv.org/abs/1810.04805 
  Repository github.com/google-research/bert
https://en.wikipedia.org/wiki/BERT_(language_model)

And in any case it's an autoregressively-trained Transformer, albeit encoder-only. If it's not an LLM, then what is it?

[–] karmickoala 2 days ago
Thank you for bringing it up. I added BERT intentionally to draw attention on this fact that I'm unable to convey properly yet: I was okay with the use of previous generation of models, so it's not that I'm against LLMs since BERT-like architectures were indeed used.

What worries me about the current generation of models is privacy. In another comment, I tried to draw the line where models start to worry me regarding it. Discussing it with whimsicalism, we noticed that it's not the generative aspect either; model size is probably the main factor. For instance, a larger encoder could make its embedding transform almost lossless and thus more prone to data leakage again, although a decoder-like makes it more susceptible.

Hacker News