Web app test automation: 50% fewer production bugs

Java

REST

Telecom

Web

Innowise has integrated automated web app testing to provide speedy and bug-free software development cycles.

Customer

Industry

Telecom

Region

Belgium

Client since

2023

Our client is a prominent communication services provider that specializes in internet connectivity solutions, modem sales, SIM card provisioning, and television package setups.

Detailed information about the client cannot be disclosed under the terms of NDA.

Challenge

Enhancing testing processes to accelerate high-quality software releases

The client’s in-house development team had initially set up a custom framework for conducting automated tests on their web application. However, this framework didn’t perform up to the expected standards, resulting in flaky tests. The instability was compounded by a lack of proper integration with the continuous integration (CI) system. This led to delayed feedback loops and increased manual intervention by QA teams.

Faced with these challenges, the client commissioned us to deliver a robust solution that would streamline their testing processes, reduce regression bugs, and accelerate their release cycles.

Solution

Advancing QA capabilities through strategic test automation

In addressing the client’s challenges, we focused on crafting a bespoke solution that would improve their quality assurance processes. Recognizing the inefficiencies and limitations of their existing custom testing framework, we set out to develop a more robust, scalable, and efficient testing strategy tailored specifically to their needs.

Framework customization and integration

Based on our experience in automated web app testing, we crafted a dual-framework strategy, employing both Cypress and Playwright for end-to-end testing. We chose these frameworks due to their unique attributes, which are well-regarded for their stability, robustness, and compatibility across various browsers and environments.

Cypress integration

Our AQA engineers selected Cypress for its exceptional end-to-end testing capabilities. One of Cypress’s standout features is its native access to browser APIs without going through intermediary servers. This significantly speeds up test execution and reduces the complexity of end-to-end tests. We customized Cypress to integrate seamlessly with the client’s web applications, enabling real-time test execution and visual testing capabilities. This setup allowed us to simulate actual user interactions more accurately. This assures that all user pathways were tested thoroughly before any code was moved to production. Innowise implemented the launch of Cypress tests within GitLab, demonstrating successful test completions on a remote server following deployment. This further confirms the robustness and reliability of our testing framework in a continuous integration environment.

Playwright integration

On the other hand, Playwright proved ideal for both front-end and API testing due to its ability to run tests across all major browsers with minimal configuration. It supports testing in headless mode, which is crucial for CI environments, and offers robust tools for handling asynchronous events and complex page interactions. We leveraged Playwright’s parallel test execution feature to run multiple tests simultaneously across different environments, which dramatically reduced the overall testing time. Moreover, Playwright’s rich set of APIs allowed us to create customized test scripts that mimic complex user behaviors and interactions. This enhanced our testing coverage and reliability. Additionally, we conducted comprehensive reporting using Allure to track test outcomes, providing clear insights and analytics from our Playwright integrations.

Strategic customization

We configured each framework to align with the client’s existing infrastructure. Innowise team introduced plugins and extensions that enhanced the frameworks’ functionality and enabled integration with other tools in the client’s tech stack. For instance, we integrated Cypress with the client’s authentication services to handle secure login during tests. We also configured Playwright to automatically capture screenshots and videos for review in case of test failures.

CI integration

AQA engineers integrated Cypress and Playwright into the CI pipeline on Gitlab. This integration allowed us to set up automated test executions that were triggered by every code commit. By automating these executions, we ensured that any new or altered code underwent thorough testing before progressing further in the development cycle. This setup not only sped up the testing process but also helped catch bugs and issues at the earliest possible stage. This helped reduce the risk of defects reaching production. The seamless fusion of these frameworks with Gitlab CI also facilitated a smoother workflow, enabling developers to focus more on feature development and less on repetitive testing tasks.

Reporting on GitLab

To further bolster the QA processes, we capitalized on Gitlab’s advanced reporting tools, focusing specifically on enhancing transparency and accountability between the AQA team and managerial staff. By tailoring these tools to generate detailed, real-time reports, we provided an invaluable resource for both teams to access up-to-the-minute data on the status of various tests.

These reports were designed to be comprehensive yet easily interpretable, including key metrics such as test coverage, failure rates, and execution times. This level of detail ensured that managers could quickly grasp the effectiveness of testing efforts and identify areas needing attention without requiring deep technical knowledge. For the AQA team, these reports served as a critical tool for tracking progress, pinpointing recurring issues, and validating the codebase quality.

Architecture development

Innowise developed a scalable test architecture tailored to accommodate a diverse range of tests, including smoke tests, regression tests, and comprehensive end-to-end testing. This architecture was designed to support the immediate needs of the project and provide a foundation for future testing initiatives. The modular nature of the architecture allowed for easy adaptation and scaling. So, as the requirements of the client evolved, so could the testing framework without requiring significant modification.

Our approach included setting up a centralized test management system that facilitated the organization and tracking of test cases, results, and metrics. This system was pivotal in managing multiple test types complexity and providing a cohesive workflow across testing cycles. By implementing robust data logging and result analysis features, we provided the QA team with the tools necessary to perform detailed evaluations of each test cycle, which is critical for maintaining high standards of software quality.

Launching tests in the Xray Jira tool

To further enhance QA processes, we launched tests using the Xray Jira tool. This integration allowed us to keep track of and manage test cases directly within Jira, streamlining communication and collaboration between QA and development teams. By leveraging Xray’s powerful test management capabilities, we maintained a comprehensive overview of testing activities. The QA team also tracked traceability from requirements to defects, and generated detailed test coverage and execution reports. This integration significantly boosted our ability to pinpoint and tackle issues promptly, driving high-quality software delivery.

Training and empowerment

Recognizing the critical role of knowledge transfer in the success of any technology implementation, we conducted in-depth training sessions specifically tailored to Cypress, given its pivotal role in the client’s testing strategy. These sessions were structured to empower the client’s in-house teams with the expertise required to effectively use, maintain, and extend the testing frameworks independently.

The training covered a range of essential topics, from basic setup and configuration of testing tools to advanced features like test automation scripting and environment management. We also included practical exercises designed to address real-world scenarios the team might face, enhancing their problem-solving skills and confidence in handling the tools.

Beyond technical training, we emphasized the importance of a strategic mindset in quality assurance practices. We guided the team on how to plan and execute tests that align with the broader business objectives. This approach ensured that the team was not only proficient in using the technology but also adept at integrating QA processes into their daily workflows.

Technologies

Backend

REST API, JSON, Java

Design

Figma, Sketch, Zeplin

Services

Firebase (Analytics, Crashlytics, Cloud Messaging, App Distribution), GitLab, Jira, Confluence, Strapi, Slack

AQA

Unit-testing + CI/CD integration via SonarQube, Cypress, Playwright

Process

Discovery and planning

The project kicked off with discovery meetings to deeply understand the client’s specific needs and challenges. This phase was crucial for setting up Cypress and Playwright, ensuring their compatibility with the client's existing systems, and preparing the groundwork for seamless integration.

Development and integration

We customized Cypress and Playwright to suit the client’s unique testing requirements and integrated these frameworks into the Gitlab CI pipeline. This integration automated the testing process, allowing for early detection and resolution of issues and speeding up the development cycle.

Training and handover

Our AQA specialist conducted comprehensive training sessions on Cypress to equip the client’s team with the skills to manage and optimize testing frameworks independently. They also provided detailed documentation and best practices to support ongoing maintenance and enhancement of the systems.

Communication and project management

Throughout the project, we maintained transparent communication via Slack and managed tasks and progress through Jira. This approach ensured that all stakeholders were consistently aligned and the project was well-coordinated and on track.

Team

Project Manager

Team Lead

Lead Automation QA Engineer

Test Data Analyst

Automation QA Engineers / SDET Experts

Results

Faster software release cycle and 2x fewer production bugs

After the successful implementation of the automated testing frameworks, the client has transitioned to a robust test automation environment that delivers significant operational enhancements.

Increased release efficiency: By leveraging automation, the reliance on time-consuming manual testing was drastically reduced, allowing for a faster software release cycle.
Bug reduction: Improved testing capabilities resulted in a 50% decrease in regression bugs, demonstrating the effectiveness of test automation processes.
Enhanced test environment stability: Through fine-tuning and optimizing test setups, we significantly minimized the occurrence of flaky tests, leading to more consistent and dependable testing outcomes.

This strategic overhaul addressed the client’s initial challenges with a tailored technological approach and expert guidance. By equipping their team with state-of-the-art tools and in-depth knowledge, we’ve set a foundation for ongoing scalability and continuous operational improvements.

Project duration

August 2023 - Ongoing

faster release deployment

50%

fewer production bugs

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.

Web app test automation: 50% fewer production bugs

Customer

Challenge

Enhancing testing processes to accelerate high-quality software releases

Solution