Hacker News new | past | comments | ask | show | jobs | submit login

Someone needs to spell out the difference between "tests" vs. "testing", might as well be me. Tests are something you, a developer, or a test engineer writes. Really important. Testing is something that the dev, a test engineer, or someone off the street can do. That is the most important part towards making working software. Unit tests are important, but are in a vacuum. The only way you can look at your code in context is to have someone use it.

Lots of games (including ones I worked on) ship with hundreds of thousands or millions of lines of code and practically no unit tests (if any). According to this article, they shouldn't be working at all, let alone make billions of dollars. Do I think it's a best practice? No. But on those projects we had an army of QA to test builds and producers obsessed with their features to always be in there making sure that things worked. The most effective testing was "everyone play the game day" (or weekend). Only then can you find edge cases that unit tests can't. Dogfooding is another important take on this concept.

tl;dr: unit tests are important in their own right, but even 100% code coverage can't tell you if the thing as a whole works as the user expects. That's "testing" as opposed to "tests".




Someone in the test industry here. I had a stint in the games industry, as well.

An army of QA is... problematic. Inevitably it turns into a death march, which is a huge waste. Worse, developers feel much less inclined to own the quality of their code (consciously or unconsciously) because "QA will catch it." I suspect that even if you wanted to be more rigorous, the incentives are against you.

Fixing bugs filed by QA is also expensive, relative to fixing the code before it's checked in. By the time a bug makes it to QA, it's a ton of patches later, the developer is working on another feature, and it's not at all obvious which patch introduced the bug. The most expedient strategy of "revert the culprit" is difficult if not impossible, and checking in new code for a fix introduces further risk.

There's certainly something to be said for expert/exploratory testing. Hell, it puts food on my table. :) But as a tester, I'm far less inclined to work on a product where developers aren't concerned about code correctness or quality even at a micro level. It says to me that they don't value my time.

Then again, I suppose in the games industry it matters less what QA thinks, given that the people actually doing the testing are often temporary/contract employees.


Lots of games (including ones I worked on) ship with hundreds of thousands or millions of lines of code and practically no unit tests (if any).

Lots of games are buggy as hell as well. In the past year, I gave up on a number of AAA games because they were simply too buggy and felt unfinished to me. Needless to say, I won't be buying sequels and will think very hard before buying from the same developers again.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: