The Importance of Manual Testing
In a world of automation, there are a few cases where the human mind still reigns supreme. Manual testing of software is one of those cases.
Manual testing, as the name suggests, is the method of executing the test cases and differentiating the actual results to the expected results. Test cases are executed manually by a human eye without any support from tools or scripts.
We reached out to a few technology leaders to discuss manual testing and why it’s so important we keep the human touch.
Joe Corrall, Test Manager, Flooid
Joe Corrall is the Test Manager at Flooid. Graduating from Coventry University, Joe got his first taste of testing when he joined Game Developer Jagex as a QA. That position started his career in Testing with Joe having testing roles at oil refinery software company Spiral before joining Flooid as a Senior QA.
Now in a team of 20+, with the majority being manual testers with a Quality Engineering Arm, Joe’s next major project is moving manual testers over to the engineering arm.
We asked Joe about his experiences with Manual and Automated testing and where he believes the differences lie.
Testing Isn’t A Tick Box Exercise
Building up a mix of manual and automated testing experience over his career, Joe is well placed to discuss the value and limitations in both.
One of the initial reservations Joe has with automated testing is around the thoroughness of automated tests that, all too often, barely scratch the surface.
“A lot of the automation work that I’ve seen done in the past is treated more as a box-ticking exercise. Someone is told they must have tests so they’ve run some tests. But, it's not really digging into code. It’s not looking for faults. It’s all very surface level.”
Instead, Joe believes firmly in developing manual testing skills that can carry over into automation testing.
“Someone who has grown as a Manual QA and moves into automation carries over a lot of good habits. They’re looking for what they can do with the software. They’re stretching it and really diving into it.”
A Machine Can’t Tell You If Your Page is Beautiful
The power in automation lies in it’s ability to clear clutter from a busy professionals day - but, there will always be limitations due to the uncertain nature of software development.
Machine learning allows the test bots to be robust and act under conditions of uncertainty unlike traditional test automation tools and frameworks.
But, regardless, the fact still remains that a machine will never be able to tell you how something feels.
“At the end of the day, it can’t tell you if your page is beautiful. It can’t tell you if it’s actually friendly to use. There will always be someone around to give it the final check. We won’t ever get rid of that.”
It’s Expected Now (To Learn Automation)
The automation testing market is expected to grow at 14.2% CAGR from 2020 to 2025. Driven by growth in the demand for automation of the testing process and solutions to seamless customer experience and rising usage of AI-enabled cutting-edge technologies for software testing environments, Test automation is quickly becoming a must-have skill.
A cursory glance over Linkedin or any job board will show you that automation is quickly becoming a ‘must-have’ in testing roles.
“It’s expected now as a QA that you’ve got that skill in your arsenal. There is still a place for a QA who wants to be fully manual but over the last 10 years it’s gone from a challenge to know to impossible to avoid. Over the next 10 years, it’s going to be standard practice.”
Paul Elliott, QA Manager, Verint
Paul Elliott is the QA Manager at Verint. Starting his career in the Aerospace Industry, Paul was able to take his experience to a Startup in Cambridge as a Tester and left, 13 years later, as a Test Manager.
In this role, Paul was exposed to manual testing and, as it turns out, a very advanced form of automated testing at the time.
“I probably didn’t appreciate it at the time but it was right at the cutting edge of automation. The size of the product and the complexity, it only dawned on me when I left that it wasn’t standard practice.”
Automation Is A Check, Not Assurance
Now leading the QA function at Verint, Paul has all of his testers running both manual and automated tests but weights the two versions of testing differently.
“My Mental Picture is that I treat automation as more of a quality check than true quality assurance. I consider a product to be done when we’ve got enough automated checks to look after the product moving forwards. The manual effort is usually front-loaded and where we find all the edge-cases.”
Consciously choosing to call them ‘automated checks’ as opposed to ‘automated tests’ so people don’t associate automation as something intelligent, Paul believes the difference between an automated check and a manual test lies in the details.
“I call them checks because they will only ever check for something that you've asked them to check for. Whereas the manual tester will go in and will notice something.”
Preferring to use automation as more of a regression backstop, Paul likes to have automation in place to support the testers throughout the projects.
The Best Automation Testers Have A Manual Mindset
Echoing similar sentiments to all the professionals who have contributed to this article, Paul believes the best testers are those who can learn automation skills while maintaining their manual testing mindset.
“If someone is just telling you about the scenarios and giving you some user stories and you go away and automate them, then you’re not really doing a testing job - you’re creating test artefacts.”
Developing an automated testing solution just a few years ago, every single tester on Paul’s team has learned automation skills in the role and, as such, benefit from a mindset a bonafide automation tester wouldn’t have.
“They look at automation in a certain way. They want to take all the manual testing and take out all the repetitive stuff, all the time-consuming stuff and script it.”
Paul, who looks for testers with this mindset, covers one topic in particular when he interviews testers: to what extent do they act on behalf of the customer?
“That, in my view, is what good testers do - they’re the eyes and ears of an organisation. They’re keeping the dev team honest and ask the difficult questions that build better products.”
Dimitri Vinogradov, Software Dev Manager, Xilinx
Dimitri Vinogradov is the Software Dev Manager at Xilinx. With a background in cell and molecular biology, Dimitri entered the IT industry as a tester around the turn of the millennium and hasn’t looked back since.
“The number of similarities between science and testing is staggering. The only difference is that in science you study nature, in testing - it’s someone’s code.”
With 7 direct reports, Dimitri has a team of testers that use both manual and automated testing methods. He believes that the two methods really shouldn’t be siloed away from each other.
The Two Inevitably Intertwine
With every member of Dimitri’s team being capable of writing at least very basic code (e.g., in bash), they’re able to stay agile and create a testing culture where no lines need to be drawn between manual and automated testing.
“Every now and then you need to automate routine activities.”
For example, you may have an output of 10 columns of text or numbers, and you only need two of them. To cut off the unwanted eight columns, you’d script something.
As opposed to automated testing, Dimitri would argue that what you’re actually doing in such a case is programmatically manipulating data - an action that is still manual.
“That's my bottom line. I don't think there is a firm boundary to be drawn anywhere between the words ‘automated’ and ‘manual’.”
Should A Tester Learn Automation Skills?
We’ve already talked about the growth of the automated testing market. Growth, for the most part, is usually a good indicator of demand for talent.
Breaking into the role of an automation engineer is challenging, and sometimes it might seem like more trouble than it’s worth. When there’s still a clear need for manual testing, what’s the point of your uprooting your current ways just to follow the trend?
For Dimitri, it depends highly on the area the candidate is working in and the rate of change happening in that area.
“Generally speaking, you automate your tests for an area that isn’t going to change much and you don’t expect any surprises from it. But whenever you’re testing new functionality, you don’t just drop everything and rush to automating tests on something that may change tomorrow. That would be twice as much work.”
Keeping your skills up to date is an absolute necessity in any line of work, and the software industry is certainly no exception. As the testing market moves more and more towards automated tests, it is clear that a tester must have at least a functional understanding of automated testing while never forgetting the core methodologies that make manual testing so effective.