Hacker News
points by annzabelle 3 days ago | 0 comments
I used to work for a company that enforced 80% code coverage for our CI/CD pipelines. Both teams I worked with there, in wildly different parts of the company, were playing games with excluding most files in the repo and writing dozens of tests that didn't actually test anything, just ran the code in the testing container so the lines would count as having been tested. I think LLMs have probably cleaned it all up a lot in the end, because the tests they write do actually test the code (against the code at the point you told the LLM to write the test a lot of the time, rather than a third party spec, but at least it's not just return true all the time).

Every 6 months there was a new testing initiative, which seemed to all be responded to by writing wrappers that returned true to the test automation system we were ordered to use.

[–] hinkley 2 days ago
I’ve seen that behavior but it usually doesn’t kick in until the mandate 90%, so that’s a shame. Usually I push back on it being a rule because if you have low coverage, you have a team that’s not good at automated testing, and forcing them to write a lot of tests before they’ve gotten good at it always ends poorly. But typically nobody listens and the shitshow begins.
[–] latentsea 3 days ago
Seems like a culture problem at both ends of the org.
[–] hinkley 2 days ago
It’s a pretty common failure mode in my experience. Keep management out of testing as long as you can.
[–] latentsea 2 days ago
Keep shitty tests out. PRs with vacuous tests should be rejected. You can have high coverage and good tests. These aren't mutually exclusive.