Peter Kartashov, Head of QA practice, EPAM Systems [NYSE:EPAM]
With the constant emergence of new technologies and global organizations focusing more on digital business, the speed of digital transformation and the reduction of time-to-market life cycles are making testing and QA even more critical. Currently, test automation accounts for over 80 percent of functional coverage for some hi-tech and innovative enterprises, while traditional enterprise is catching up with the overall market expectations of over 60 percent test execution coverage by 2020. And so, we know that the phenomenon of software test automation over the last two decades will only continue to grow. Looking back at the history and evolution of test automation can help us understand what the role of testing engineers will be in the future.
Industrial test automation started from hardware, where it was embedded into its operation mode. The process was quite similar to what we use in unit tests, in well-designed and properly managed software, except that this diagnostic-level testing was built into a live system as a required step prior to launch. The next evolution of test automation gave birth to toolkits, such as Rational Robot and Mercury QTP, and during that time, the predominant focus of testing was on the UI level. New testing COTS vendors began to emerge with better and more user-friendly functionality, which resulted in market fragmentation and many unmet expectations about automation. Over the last 10 years, emerging hi-tech software companies began to understand that the only way to build sustainable automation, while producing their software quickly, was to build it in parallel with their products and share the same technology stack as the product itself. With the growth of Java and open source, many developers and automation engineers turned to building custom harnesses from scratch by combining various open source or freeware libraries.
Today, the industry has entered a paradigm shift with a new breed of cloud-based and continuously released products. The process of building software, the infrastructure in which it resides, and the product itself are so cohesive that we consider them as one indivisible asset. As such, testing is now both a process driver and part of the product assets, where we have to mobilize a full-scale quality engineering practice. Ironically, looking back at where we began—with testing and diagnostics built directly into operating firmware—we are now experiencing a similar shift in the engineering of modern software.
The adoption of cognitive computing and machine learning will bring even greater opportunities to testing and deployment automation
By embedding automation into the product and stretching it across the entire product lifecycle, we add a shift-right component to production software support, such as user feedback, production analytics, diagnostics, and self-tests, in addition to shift-left testing during active product development. The emergence of continuous testing and the role of a Software Engineer in Test (SET), however, began in 2008, but it describes the approach as displacing manual testers and introducing a new role called Software Development Engineer in Test (SDET), pioneered by Microsoft and then adapted by Google as the SET role. However, the modern cloud-based implementations that demand a continuous testing process work better with the SET approach.
The role of an SET entails building a robust testing process into the product by offering full stack quality engineering skills that, in addition to traditional test automation, bring some hybrid functions such as:
• Co-engineering and co-managing with Dev leads and architects on coding best practices, conventions, and code coverage traceability, along with the rules for both dynamic and static code analysis; the SET role also brings technological integrity between developers and test automation engineers by ensuring that these engineering practices are aligned and used accordingly
• Leveraging past data and real-time analytics to build user-centric test data and challenging product usage scenarios
• Using general-purpose (such as AWS) and specialized cloud (such as mobile cloud, Selenium-in-cloud) capabilities to build complex execution strategies across various browsers, devices, locations, mobile carriers, and contexts of use, in order to maximize integration and compatibility test coverage
• Promoting and incorporating the shift-left definition of “done” and the SDLC process by helping teams establish right product testability from both the process (how the product owner defines user stories and acceptance criteria) and the technology perspective (how underlying product architecture accommodates testing and integration)
• Assisting and coordinating with DevOps engineers on building CI pipelines and automatic quality gates with clear reporting, in order to achieve a continuous delivery process
• Constantly finding and coordinating optimal equilibrium of test coverage pyramid among unit, service, UI/UX, and end-to-end integration testing levels; in addition, by leveraging the power of established traceability, SETs help to build efficient test execution suites based on potential impact/probability, as opposed to full regression execution every time
• Harnessing performance and reliability testing tools which are ultimately incorporated into the CI process
• Building a quality engineering culture across the entire product lifecycle from product conceptualization to ongoing live product support to have full traceability from high-level product definitions to individual lines of source code
Practically speaking, digital businesses will require faster reaction to user expectations and market competition. SETs will be key in building highly adaptive, always-on testing machines. Furthermore, the adoption of cognitive computing and machine learning will bring even greater opportunities to testing and deployment automation. With that shift, the industry will likely move to the next evolution of testing automation, where high-end engineers with “quality first” mindsets will implement a testing-for-quality process.