Beyond core test activities, Gen AI can play a critical role in integrating tools (especially with the CI/CD or DevSecOps pipelines), optimizing test dependencies, enabling faster test execution cycles, reducing redundancies and roadblocks that can hinder productivity.
Gen AI can also help clients modernize their tech landscapes and test architecture faster. While traditional approaches might take weeks or months, Gen AI can enable autonomous script migration from older to newer tech in a fraction of the time. And because these capabilities are easier to use through generative AI than through traditional AI models, teams may not need to invest as much time and money in building technical skills to adopt and implement an AI program.
Synthetic Data Generation
Synthetic data generation is one of the most effective and low-hanging use cases for generative AI. To ensure adequate test coverage, QET teams need adequate data. In many programs, this can be difficult to obtain with production data alone due to challenges with regulatory compliance or insufficient data masking. QET teams turn to synthetic data generation to supplement production data and acquire test data for scenarios the business has not yet encountered but is likely to encounter in the future.
For example, in the insurance industry, policy and claims systems require information on customer demographics such as age, gender, location, prior health issues, to recommend the right insurance policies. Because the data combinations needed for testing such scenarios is exhaustive and extremely complicated, production data alone may be insufficient, leading to gaps, oversights or skewed testing results. Generative AI can help produce the necessary test data to supplement production data.
It’s important to note that while generating synthetic data using large language models with generative AI may make sense for early-phase testing, this approach will need tweaks to make it cost-effective for large-scale projects due to the tokenization Gen AI uses to monitor and monetize data. Companies may consider hosting their own Gen AI model or infrastructure or incorporating alternative approaches such as generative adversarial networks (GANs).
Modernization and New Technology Adoption
To keep pace with technology advancements and beat off competition, many banks have been acquiring fintech companies for specific processes, such as retail loan processing, and migrating only those specific processes. Decoupling features like this allows banks to adopt new technologies and modernize faster because they don’t have to overhaul the entire core system. Still, these changes require major reworking from a testing perspective. The bank would need to modify the loan-oriented test cases, assess the impact of changes, create new test cases for modifications in the workflow, grade new tests for the new product and match release cycles.
QET teams can use generative AI to expedite these tasks by prompting Gen AI to generate test cases, scan code for anomalies and predict potential issues. Advanced natural language processing of generative AI can help streamline technology migrations by assisting with translation. If a bank is migrating from a mainframe to a newer Java-based system, for example, the process typically involves migrating technology as well as testing assets. Testing teams can feed the old scripts or assets into a generative AI platform, which will then translate those scripts, speeding up the process significantly. Test teams can also use Gen AI to automate these tasks for greater efficiency and ongoing refinement.
Improved User Experience
Generative AI’s sentiment analysis is better equipped than traditional AI to analyze text such as customer reviews and emails and pick up on the nuances of language. Traditional models struggle to identify sarcasm, for example, and might mislabel a negative review as a positive one. Gen AI, on the other hand, can make those distinctions. When combined with other AI capabilities, models can even trace those comments to potential root causes in an application and suggest corrective action.
Generative AI also helps increase observability in the production environment. Securities and capital market (SCM) organizations, for example, are always on the lookout for quality interventions which can improve test coverage holistically for their trading platforms. Gen AI can help autonomously reverse engineer customer journeys into UI test cases and automation scripts. It can also analyze API logs and autonomously generate test cases and scripts relating the front-end user journey to the relevant navigations through the back end. These capabilities ensure that critical financial transactions are not impacted, which could lead to heavy regulatory fines.
The Importance of Validating Generative AI Models
Due to information security restrictions, many organizations are planning for either open-source models or building their own large language models. But before any organization can incorporate generative AI into their testing programs, they need to be sure the model is secure, reliable and compliant with regulations. This requires a thorough validation process that considers the various challenges and characteristics of the model such as biases and errors, complexity, and scale. The validation process should clarify the intended use of the model and what is needed to support those functions, help develop tests to evaluate the model’s performance, compliance, and suggest corrective action.
Validating an AI model and supervising the outcomes given by the model is an ongoing process that requires a robust quality engineering strategy to ensure constant monitoring and updating based on changes in performance, business goals, industry demands, regulations. Engaging stakeholders throughout the organization to work with the quality engineering team can help develop a stronger program that meets the shared goals of the firm. For larger, more ambitious projects, a technology partner can help by working with the various stakeholders to align on a vision for generative AI in the business and develop a realistic implementation strategy.
The true value of generative AI is likely to vary from business to business depending on the specific needs of the organization, its employees, and its customers, but the potential of this new technology cannot be understated. Ongoing investments are leading to rapid development, and savvy enterprises are finding new opportunities to combine Gen AI with other advancements in automation, data analytics, IoT and cloud. This is a boom time for innovation, and QET teams can make the most of it by exploring Gen AI strategically, with business value top of mind.
