Hacker News

points by fny 3 days ago | 0 comments

"These chat transcripts may even be read by human reviewers."

PSA: This is true of all the frontier model providers even if you disable activity tracking. They are all constantly monitoring your interactions for problematic behavior.

[0]: https://www.nytimes.com/wirecutter/advice/ai-chat-privacy/

[1]: https://www.404media.co/inside-project-lily-the-humans-readi...

[2]: https://www.winknews.com/news/woman-arrested-after-ai-threat...

[–] gkcnlr 3 days ago

The sole reason LLMs has gotten this good is because their aggressive expansion has been accompanied with literally this. The amount of uncredited information these models have been fed must be astonishing, also terrifying.

[–] synalx 3 days ago

I think the heavily subsidized subscriptions are exactly for this purpose - the training data they generate is worth the high subsidy.

[–] jpardoe 2 days ago

They're almost certainly making a profit on subscriptions.

[–] cj9 1 day ago

Right, but they are also certainly looking to make higher profits overall by avoiding having to pay for large amounts of (confidential and/or private) training data. Most of what they have would never be for sale either.

[–] jpardoe 1 day ago

Seems smart.

[–] red-iron-pine 3 days ago

i mean openai basically said as much, no? it's free for all so that they can train the models better and get real world interaction data.

[–] cma 3 days ago

Yes, Altman called it the data flywheel in 2022.

[–] terramauthe 3 days ago

yeah, but in 2026 the interaction data of the average user isn't adding much to training. only the really exceptional users.

[–] cma 1 day ago

They can filter for the exceptional users, but the data flywheel is also about getting access to private codebases, not just the interaction in the chat.

[–] MisterMunchkin 3 days ago

Why do you think they’re getting into consulting now?

Imagine all those tasty confidential IP and contract documents they have ingested. You couldn’t buy those.

[–] cududa 3 days ago

That is, fundamentally the marketing ploy of “AI Safety”.

For an AI system to be “completely safe”, like a self-driving car, it must map the full environment it operates in and be taught the “right way” to handle every conceivable scenario.

So when you present it with novel data or a situation, that constitutes an “AI safety event”, which per their ToS is a carve out for training on your session.

Further, go read the ToS carefully about what you opt out of. You’ll note there’s nary a mention of tool calls. You’re only opting out inputs (what you type) and outputs from the model from being trained on. All the content of the tool calls made while running on your system are fair game for training.

[–] eckr 3 days ago

There is no way this is true, no? This'd mean literally all codebases people are working with have been trained on. There's no way anyone would use these guys if that was the case?

[–] bethekidyouwant 3 days ago

Why are you using it if you don’t want it to get better whose data is it supposed to get better on just somebody elses, not yours?

[–] popinman322 3 days ago

They're supposed to be focused on genuine training improvements over existing data, including via RL. Gathering data under the table is misaligned behavior.

[–] degamad 2 days ago

> including via RL.

Can you please expand on what you believe RL looks like in this context?

[–] alberto467 3 days ago

Well for us Europeans, GDPR explicitly allows them to store pretty much anything if somebody can claim it’s for safety.

It’s literally just complicating things and providing zero reassurance to us AI customers.

[–] tveyben 3 days ago

For safety - hmm new for me (European), please elaborate.

[–] manmal 3 days ago

You don’t need user consent to retain data that’s arguably needed to provide the service itself. I guess, safety falls into that category. Training on that data is probably a grey area, if they can guarantee it won’t contain PII. They have stolen all the internet, so I wouldn’t bet highly on their restraint.

[–] Dylan16807 2 days ago

It's very easy to prove that training is not necessary to provide the service, because whenever you're using a model it's static and the training only maybe goes into new models.

[–] zombot 2 days ago

It doesn't say for whose safety. It's certainly not yours.

[–] zombot 2 days ago

Additionally, how stable are those ToS? Do they change every few days, like their models?

[–] fierycatnet 3 days ago

I can guarantee you that multiple people are reviewing some chats. This is very similar to Apen and whatever company prior to that for reviewing Google searches. They'd hire out contractors with NDA.