Hacker News

points by tikotus 2 days ago | 0 comments

I can believe this is true. It's great at going out there, connecting ransom dots in very clever ways, sometimes stumbling on something actually funny.

I'm curious, did you test with the 200 jokes if it could guess which were the beat ones according to a human audience?

In images, music, games, it usually can't. So the process of validation is offloaded to the audience. I'm very interested in seeing where this takes different platforms. I bet Spotify can pull off such a validation process. Music is passive, tolerating a bad song doesn't require much from you. And they have a massive audience. They can try out AI generated mess with a very low success percentage without saturating their playlists and alienating their users. But can Steam tolerate it? Trying out a game is a much bigger effort than listening to a song picked for you. Customers are already reacting to the number of demos in Steam Next Fest. The saturation point is much lower.

And what about jokes? What happens to r/jokes if even 10 standup comedians are validating 200 jokes per day?

And we see it in software too. QA is already being offloaded to the customer.

I want to see where this goes, even if I don't like it.

But if you find a media where AI can also validate the output instead of having to offload it, that's going to be very valuable!

[–] pcwelder 1 day ago

It matches your experience. Its judgement on which joke will get laughs is only slightly better than random with lots of false positives.

[–] antonvs 1 day ago

I wonder how much different that is from humans, even professional comedians. You don't usually get to see all the jokes that comedians try that don't work, unless you're an avid attendee at comedy clubs.

[–] tikotus 1 day ago

I'd guess the difference is around a factor of 100 or so.

[–] antonvs 1 day ago

For comedians, perhaps. But there are plenty of humans who aren’t very good at comedy.

In these LLM comparisons, people tend to compare them to the best humans, not average or below average. It’s a kind of motivated reasoning, with the goal of deprecating the abilities of LLMs.

[–] tikotus 1 day ago

Well, you did ask for a comparison to professionals.

But I think humor is pretty easy to rate for a human. Did it make you laugh? Yes? Then it might be a good one. This simple human filter radically improves your efficiency as a joke validator, leaving any LLM far behind.

For pictures I'm not sure if humans are much better. For music, yes, after a few listens (recognizing a hit usually takes time). For games, definitely yes.