This is the next instalment in our series of articles on the Technical Questions To Ask Your Software Development Partner. All of the 3 questions we’re looking at in this series of articles are important questions to ask external vendors, internal candidates and external contractors. We start with the first question to ask:
What kind of testing quality controls do you perform when you release some software?
As a software development studio that is often called in to rescue a custom software project gone wrong, we see a range in the quality of testing that has existed in the project.
Hey it compiles – ship it!
At the very bottom end of the scale, we see development where the quality of testing is “Hey it compiles" and then they'll ship the code. The problem with this approach is that just because something compiles, it doesn't mean that it's doing what it's supposed to be doing and doesn't mean that it's going to work in different test cases.
If your current software development partner stops at the point of “Hey, it compiles. It's good to go!”, you're going to have problems with the software in the end as it's not going to be particularly robust.
Is TDD the answer?
On the other end of that scale, you get developers who will aim for 100% unit and integration test coverage, maybe as part of test-driven development (TDD). Depending upon the size of your budget, that can be a way to go. There are organizations out there, very big ones, that use this very successfully.
An example would be Netflix. Netflix pretty famous for their testing methods and the way that they handle their releases.
They randomly attack their own network and randomly switch off servers because they figure that, if their infrastructure cannot cope with that, then they've got a problem that they need to fix.
Now, the kind of tests that Netflix do is really off the charts, and very few people strive for that. But what I have seen is that excessive or compulsive attention to completing 100% unit tests, 100% integration testing can actually add to the technical debt of the platform or application you’re building.
The reason for that is that, if you need to make a change and you're going to end up breaking a whole bunch of tests, then in order to make that change, you've actually got twice or maybe three times as much code you have to change, to be able to do that. That's where the technical debt comes in.
Now it’s not to say that your software development partner shouldn't be using unit and integration testing because, ideally, they should as you will end up with a more robust product. As with anything in life the trick is to get the right balance.
I don't think that 100% coverage is a good thing to aim for, what we tend to do is aim for enough coverage that it's going to catch cases where functionality could change in the future and you need to be aware that and alert to that when you're working on the code rather than later on down the track when you've put it in production, and then you realize “Oh, hey, look! That calculation has changed and we're no longer collecting the correct amount of tax.” (I don't think the tax office looks kindly at that)
What should I be looking for?
Unit testing, integration testing, and ideally also manual quality assurance. Some kind of manual QA is a very beneficial thing to have on a project, just ensure that you aren’t entirely reliant upon it.
The two together (automated testing and manual testing) work well to be able to deliver a more robust product. The end result of doing this and having these processes in place is that you'll get functional code.
Caveat: An important distinction to make here is that just because code is functional does not mean that it's high-quality code. It means that it should do what it is intended to do. And the two are not quite the same thing.
You want to get a solid answer on the types of testing being done, looking for unit testing, integration testing, and some kind of manual QA. If you're asking this question and you don't get these kinds of answers then that’s a red flag that should prompt you to investigate or to seek to get your software developed elsewhere.
If you’ve got a software project idea that you’d like to discuss, CLICK HERE to book a time for a scoping call to run through your idea.