Hacker News

points by anonymous908213 3 days ago | 0 comments

> pgrust shows the way.

You mean the pgrust which was recently caught for cheating benchmarks and incapable of even performing basic queries outside the benchmark set? Would love to see pgrust debunked by Jepsen like this other AI-generated, written in Rust, "formally verified" scam.

https://jepsen.io/analyses/particledb-1.1.4

[–] est31 3 days ago

I think the pgrust controversy should be resolved now?

https://news.ycombinator.com/item?id=50209450

[–] anonymous908213 3 days ago

I don't consider a cheater downplaying their cheating to resolve the fact that they cheated, no. Every cheater has a defense when caught, and theirs was not convincing. At this point I think the only thing that could redeem their brand would be paying for a Jepsen analysis to verify pgrust is as good as they claim it is; I have zero reason to trust the words from the horse's mouth.

[–] malisper 1 day ago

You don't have to take my word for it. The ClickHouse CTO (who opened the original removal PR) updated his post after reviewing our explanation and our fully open-source v0.4-beta submission[0]:

> We clarified with the author - while the binary had signs of optimization for specific benchmark queries, we have a reason to believe that the actual runtime didn't use per-query kernels... The usage of a query condition cache and cached statistics per granule is legitimate. The submission of a closed source preview version fine-tuned for benchmark was shady nevertheless. However, the new submission #2751 should move us post all the concerns.

Submitting an incomplete, closed-source preview build was a mistake on my part. Since then, we open-sourced a new version with full SQL support, ClickHouse merged it back into ClickBench, and the new version actually runs 7% faster (geomean) than the preview build that was removed.

[0] https://github.com/ClickHouse/ClickBench/pull/2165