Hacker News
points by fscaramuzza 3 days ago | 0 comments
My PhD is on auditability of AI systems, and we are exactly working on the engineering of ZKP in this context. I can confirm through our industry partner (AI Assurance and auditing), that black box auditing is indeed a huge problem for them, and snarks have shown their potential in this context. It's great to see Anthropic confirming our early results.
[–] ngruhn 3 days ago
I'm blind to the problem being solved here. Can you motivate it? Why do I need more than normal HTTPS on the token API?
[–] fscaramuzza 3 days ago
HTTPS makes sure you’re communicating on a secure channel with the API endpoint. ZKPs guarantees that the computation happened correctly behind the endpoint.

The work from Anthropic isn't a new idea, but they made the process more efficient, as inference verification is one of the biggest bottlenecks in the AI lifecycle. Beyond that, there's a lot more that can be proven, like showing a model was trained on a specific dataset with certain properties.

From an auditing perspective, this used to be handled either by showing the data (which is pretty rare), checking samples (which has been shown to be easily hackable), or similar workarounds. For inference, an auditor might want to interact with a model on demand and have absolute certainty that the model they're talking to is actually the one declared, and not a substitute.

[–] grey-area 3 days ago
I’m not sure what you mean by correctly here - do you mean with the model expected? That’s not related to correctness.

Pleased to hear somebody cares about this. What is it a bottleneck for though? Are people actually doing this right now in non-academic settings? I had the impression this is not in fact usable with larger models.

[–] fscaramuzza 3 days ago
ZKPs verify "things" by encoding the verification in a computation, and verifying that the computation happened in the right way, means to assess it was computed correctly, i.e., with the right inputs, the right operations, etc. So this is what I meant with "correctness".

The bottleneck has two sides: first, there's a technical bottleneck. ZKPs are quite expensive, and finding the right, production-ready protocol is challenging. The other side is a socio-technical bottleneck. There is always quite a large friction with adopting these kind of technologies, also due to the first bottleneck.

As you said, it is difficult to adopt because of how fast the complexity grows with big models. It's great indeed to see that Anthropic is tackling the challenge.

[–] ngruhn 3 days ago
> guarantees that the computation happened correctly behind the endpoint

but why do I care about that? can you some example "user stories"?

[–] brcmthrowaway 3 days ago
So, you got beaten to the punch?
[–] fscaramuzza 3 days ago
Well luckily not really, their work is more foundational-y, the idea is not new, but they made it more efficient and practical. On our side, we work more on the integration of these protocol in production systems, but we are pretty much on the frontier and is not very common to see such big companies experimenting with this technology. Microsoft already tried something [1] but I'm not sure if it ever went further than a research prototype.

[1] https://arxiv.org/pdf/2402.02675