Reflecting on My Quality Assurance (QA) Journey

This article’s title must seem really out there for those in an industry that relies on engineers, tech-savvy analysts, and the like. These people have grown to rely on robust coding-based automation suites to ensure their releases are as bug-free as possible, especially in an industry that demands constant improvement and as little interruption to their users as possible.

When humanity went from flying around in planes to landing on the moon in a relatively short amount of time, we were in absolute awe at the possibilities that we could achieve. As someone who didn’t have a coding background, I saw the transformation in e-commerce automation testing in the same way—a giant leap for an industry that required a massive change to how we go about making the website/software industry easier to perfect. 

I understand that the comparison between landing on the moon and no-code automation is not on the same scale. However, the website/e-commerce industry has only been around for about three decades—a relatively short time to switch from a coding-based approach of ensuring your site works when it gets released to the market to a codeless test automation tool that the average person can use to achieve the same at much lower costs to the business.

Before entering the e-commerce world, I held many unrelated positions: retail, banking, and teaching. The only “testing” I did up until that point was for the banking industry, and it was minimal. In my personal life, I’m a PC gamer, which, as far as I’m concerned, is valid computer experience. I also rarely shopped online–I still mostly shop in person, although that’s changing over time.  

As a Gen Xer, I grew up in the computer age. I knew DOS, then moved on to the first version of Windows, and finally, became a macOS user while working in the industry. I never would have imagined working in this domain due to a lack of coding knowledge. I always thought testers only existed for the gaming industry. 

When I started on my journey as a Quality Assurance (QA) Coordinator, the job seemed really simple: 

1. The client needs our agency to develop/create a page on their website or an entire website from scratch.
2. Our team reviews the requirements, such as design, functionality, and integration with other components.
3. The team develops what the client wants and “tests” the product locally before releasing it into a test development environment.

This is where I come in. I have a simple list of test scenarios to verify: 

• Does it look like what's in the design? 
• Does it function like it was supposed to? 
• Did we break anything?
• Does the checkout still work?
• How does it look on this device/browser? 

So many questions, so little time! 

I quickly discovered that developers dislike testing almost as much as they dislike testers—“What bug? It’s fine!” Don’t get me wrong; we get along quite well, and over the years, they’ve become a little better at testing their own work.

As you’re probably guessing, I started out testing our sites manually. To avoid massive costs related to keeping enough variety of physical devices around to test on–which are always constantly changing anyway–we chose to go with BrowserStack, a low-code/no-code app and browser testing tool. This allowed our small agency to ramp up our testing to our handful of clients fairly quickly and allowed me to learn the ins and outs of the e-commerce world in the process. 

At first, I had to get used to testing the Front-End (FE) on the various sites to better understand how to be a proper QA. I learned how to log bugs, how to communicate them to our various FE developers, and what to look for when we were ready to go live with the completed task.

Most QAs will tell you that the job requires not only attention to detail but also a massive amount of patience. You’re basically doing the same test repeatedly on different devices/browsers. At first, this doesn’t seem overwhelming when you only have a few sites to deal with; however, as the business grew, this became quite staggering for a manual QA. 

Now, add all this testing to the Back-End (BE) of all these sites, and the amount of QA required to ensure stability became overwhelming. Although it made the job more interesting, and I learned quite a lot, the amount of QA required to achieve constant stability was massive. 

There’s only so much a human can do without forgetting something that has the potential to break the most important function of any e-commerce site: the checkout. This removes the ability of the end user to purchase anything from our clients, which could be catastrophic.

As our agency grew organically with the number of clients we had, so did our QA needs. We continued to QA manually, and although we did well to catch issues, we were never perfect. In an industry that demands perfection–or as close as you can get to it–there’s only so far you can go with manual QA.

The Search for a Codeless Automated Testing Tool Begins

The trouble often comes when the lifecycle of releases creates bottlenecks. Basically, if every website you’re working on needs new code released every week or two, that means you have less time to QA than ever–especially if you’re covering multiple sites with a really small team. 

Luckily, the market realized that there was a desperate need for automation that did not require any coding experience. For many small agencies, automation was likely not within the budget since it required an experienced developer and a mountain of hours to create automation processes and suites for more than one e-commerce site, making the task all that more complex and time-consuming. 

When Jon, our Chief Technology Officer (CTO)/Owner, told me that a no-code QA automation tool was available and could solve our QA needs, I was ecstatic beyond words. We discussed the strategy for our most time-consuming and sensitive QA issues and decided on the best way to move forward.

When I started working with the first no-code automation tool we selected, I had no real expectations or idea of what to look for, such as ease of use, test stability, maintenance, the length of time it took to create automated tests, and the support it offered. At first, it was an interesting experience learning a new tool that would hopefully help our agency build a more efficient QA process, but as time passed, it proved to be a hassle.

I’ll admit, it was nice to start with: click to record your journey, make assertions along the way, and let it run on the cloud to see your results. At first, I thought it was normal to have to write some commands or for the cloud to take a while to run on its own. 

It all seemed easy, fun, and a great way to automate away our pain points. What we found, though, was that it was not as promised, and it ultimately turned into a very frustrating experience.

The biggest issues that I faced were test stability and support from the company that provided the tool itself. It took a long time to run and create the tests, and once you thought you had a stable test, it would fail on subsequent runs. Support was not very helpful, and we even reached out to the CEO as our problems seemed to never have a logical solution for a tool that was promised to be one of the better ones on the market.

We tried working with their engineers' solutions our issues went beyond their normal support program, but it cost us more time than it saved. The whole point of this was to save us time, not waste it on trying to solve issues that this tool was supposedly designed to do.

We didn’t give up, though. We knew there was something out there that would be a better fit, and we found it fairly quickly.

One Giant Leap for all QA-Kind!

When you get the right tool for the job, it feels good! It’s amazing when things fall into place, and everything starts to move in the right direction really quickly. I felt like I was on a spaceship on the way to the moon—from the “caveman” ways of manual QA to using a no-code automation tool that would help our company move ahead by leaps and bounds. 

After a month of testing, we narrowed it down to 3 tools and found one that stood out from the rest - Testim by Tricentis. They had it all:

• Ease of use/user-friendly interface
• Test stability
• Easy test maintenance
• The length of time to build tests was short
• Amazing support
• Helpful documentation and even better examples of use cases

Once we found the best no-code automation tool for our needs, we built test suites to alleviate our most important QA problems. Within a short amount of time, we were able to cover a few very important clients with short release cycles.

This tool provided our agency with some new advantages that we didn’t quite have beforehand–a significant assurance that new code wouldn’t destabilize steady conditions on the sites and the ability to cover 100% of the test cases that we required to provide the certainty that our clients needed.

Now, before our releases, we can run our test suites overnight in our test environment. The next day, a review of any failures is made, and in most cases it catches what we wouldn’t be able to if we were to manually QA the sites. 

This is because previously, there were too many test cases to provide 100% test coverage, as we could only do about 25% before each release if we were lucky. Now, with this added capacity, we can save time, cover a variety of tests, and learn even more about QA items that aren’t necessarily covered in the automation suites, such as visual defects, and Back-End testing.

This has helped us reduce risk, improve our overall release cycles, and maintain a trusting and healthy relationship with our clients. In addition to those advantages, we now have something we didn’t have before: an extra pair of “eyes” that doesn’t miss even the smallest change that our developers may make that could have been missed with our manual QA abilities. 

As much pride as I take in my ability to catch bugs, this tool’s robust testing helps us catch anything that may have been missed previously. 

Productivity, Efficiency, Predictability - Make it Part of your Business Strategy

When I started as a QA over seven years ago, I couldn’t imagine that I’d be using a QA automation tool to do the job. Technology has moved quickly in our lifetime. From the good ol’ days of dialup with visually horrendous websites to ultra-fast, user-friendly sites offering a great user experience. 

We went from testing desktop-only to testing on tablets, and now we focus primarily on mobile devices since the world mostly uses handheld devices to access what was previously reserved for the few who could afford a home computer. 

Once you start working in a technology-based field, especially in the fast-moving industry of website development or online shopping, as I do, you start to appreciate any edge that may come from emerging technologies in the field. 

Since we started using codeless automated testing to help us QA our clients’ e-commerce sites, the benefits have been clear: stability, predictability, and a stronger trust in our ability to release code that doesn’t break their customers' ability to use the site for all kinds of purposes.

Introducing a low-code/no-code test automation tool to your business may be hard if you're a small agency with a tight budget for your QA needs. Alternatively, it could be extremely cost-effective if your company employs the more expensive strategy of using engineers to code all your automation suites. Either way, I believe this is the way forward before artificial intelligence takes over everything we do. 

When you’re ready to start researching the plethora of tools currently available on the market, here are some things to consider:

• Determining who will primarily be using the tool. Will it be used by an employee who doesn’t know how to code or a developer with a more in-depth knowledge of code who will use the tool differently than a non-coder? There are tools out there that are tailored towards the former and some for the latter.
• Figure out how many iterations of tests you’ll need to solve for your business's most important pain points on the atomic level. This will inform your budget for the tool you choose, as many of them are based on how many monthly tests you run on their cloud. Also, see if local runs are counted towards the total for the month and how many servers you may need to use to run the tests on the cloud.
• Make sure the tool can perform some custom actions, such as API calls and custom JavaScript. You never know if these lines of code will be used later on for more complex tests that a developer can integrate for your QA team. We’ve used it to assert prices on B2C/B2B products, pull gift card codes for testing checkout scenarios, and extract information from emails to ensure the correct information appears every time.
• Not all tools work with the system your business is using. Some of them are exclusively for Windows platforms, so compatibility may be an issue if your business uses macOS. Also, certain tools don’t have a way to test on mobile devices. Keep that in mind if it’s a requirement.
• When your business goes through a Proof of Concept (POC) with one of these tools, build the more complex type of tests your business would need first to assess how this fits into your requirements. During this discovery period, you’ll really get to understand the limitations and see if it’s a good fit.

Once you select your new low-code/no-code automation tool, here are some of the best practices I’ve learned over time:

• Make sure all of your QAs are involved in creating tests. As your business grows and turnover naturally increases, it’ll be important to have as many people with in-depth knowledge as possible to train the next new hires. 
• From time to time, a developer will be needed to get involved with some of the tests, such as API calls, custom JS for complex tasks, etc. Although the tools are used without requiring any code, there are times when some are useful for more complex tests that you might want to perform.
• Make your tests simple, short, and organized. The longer and more complex a test is, the harder it is to fix or maintain. As you get more familiar with the tool, you’ll want to ensure you use your time with it most efficiently. Group similar steps into shared folders for reuse across different tests, and modify only when necessary if something changed that would affect all tests that share this step.
• Reach out to the tool’s support team to find out if they have more efficient ways to improve performance for your various tests. They know the tool better than you and may have great recommendations to help you achieve the same results in a more efficient way.

Finally, I believe this is the future for most businesses, big or small. No-code automation will ultimately be the cheaper option, as it doesn’t require someone with coding skills. As our world becomes more technologically integrated, we’ve devised ways to make creating and testing applications simpler. Some may see this as an unnecessary expense for their business, while others will see the value and cost savings in real-time.

To summarize, this is why your company should consider this a codeless automation tool:

• Productivity: Once you create a suite of tests for your biggest and most complex processes, your team can concentrate on different aspects of your business, freeing up time for other initiatives and allowing your release cycles to go out without any unexpected delays or major bugs. Since this is used to catch bugs quicker, your devs will be able to have issues fixed well before your product release date to fix it. Therefore, they’ll also have more time to work on new initiatives and development without as much hindrance. 
• Efficiency: Instead of either manually running the same suite of tests or creating/maintaining complex code to cover the tests, your team will only have to review your test runs if any failures happen. After that, it’s really easy to either maintain your tests if your application changes or simply create new tests based on new requirements. Since these can run overnight or on weekends, your team can do more during the day to meet those tight deadlines.
• Predictability: In this line of business, where technology constantly changes, predictability is a huge asset. If your release cycles are more stable and less buggy, your clients will be happy to have a reliable partner. There’s nothing worse than testing until the last minute, only to find a new bug or, even worse, discovering it after it’s been released to the world. No one wants that, and if your testing can predictably release mostly bug-free applications, it will only increase your bottom line. This will build trust and a long-lasting relationship within your business circles. 

Conclusion

This giant leap from a coding-based automation that was mostly unaffordable for smaller agencies to a non-coding-based automation tool that requires very little training and expertise has assisted our type of business to grow. Not only is it more interesting for your QA’s to use than manual testing, but it also allows you to redirect their efforts to more complex projects that were not being properly tested by your devs (they’re not all at fault, or are they?). 

For now, this is the leap forward that we need - until Artificial Intelligence eventually does it all - a leap for your business. Your only regret will be, “Why didn’t I do this sooner?”

Asaf Salem is Blue Badger’s Quality Assurance Team Lead. With seven years of experience testing the sites our agency’s clients rely on daily, Asaf prides himself on his ability to lead and motivate his team to continuously learn and improve to help our e-commerce clients grow and thrive in this competitive–and constantly changing–space. 

Blue Badger is an e-commerce agency offering services like the design and development of custom themes, strategy, conversion rate optimization, performance marketing, and more. Let us help you build an e-commerce ecosystem that you can rely on. Get in touch with us today to learn more.