Companies are looking for new ways to deliver high quality and reduced costs amid a dynamic business environment and unrelenting financial pressure. These challenges are compounded for pharmaceutical companies, where software testing consumes additional resources due to Guidelines for Pharmaceuticals (GxP) and Sarbanes Oxley (SOX) compliance obligations.

Any project or change containing code and configurations carries inherent business risk, as each solution will potentially impact key business processes. Fortunately, pharmaceutical companies using SAP applications can meet corporate and regulatory objectives while mitigating these challenges using SAP Test Factory.

Why SAP testing is fraught with challenges 

As businesses become more app-driven, there is increasing emphasis on testing packaged applications like SAP to ensure all customizations and extensions are not impacted by new software releases and/or updates. In the case of SAP customizations, business process testing needs to start with vendor software standard capabilities and then extend to cover customizations like SOX compliance.

Four issues can obstruct the SAP testing lifecycle:

1. Need for Strong Test Governance & Management at the Project Level

Large organizations always have a number of projects running in parallel with mixed methodologies, some following waterfall, and others agile. Each project deliverable needs to meet specific quality gates that are defined at every stage. For pharmaceutical companies, internal and external regulators place an obligation on the company to be able to demonstrate, with evidence, that all regulatory requirements, including GxP requirements and any SOX regulations, have been met. Companies must:

• Provide evidence of the approach taken for testing, or justifying where testing is not being conducted
• Describe the actions required to achieve the end results
• Produce evidence of what was authored, what was executed, and what was evidenced for the correct testing and validation of the requirements

For each requirement, a dedicated Test Manager must work as the quality gate on a project. Most organizations work with several different partners in their delivery capability, which further complicates the process.  Consequently, independent strong governance for all projects, with streamlined processes, is required for project success.

2. Test Manager with SAP Knowledge

There are several quality gates in the overall project lifecycle, some related to financial control, others related to GxP, etc. Most are from a process and coverage perspective, but it is very important that the Test Manager has dual knowledge of SAP and quality gates. The Test Manager should be able to validate quality gates, work as an effective interface between project and compliance teams, provide technical insight regarding changes, and guide the project team through test lifecycle activities. A Test Manager without SAP knowledge slows down the entire process and becomes more of a layer than an enabler.

3. High Cost of Regression Testing

Parallel changes from multiple streams or projects necessitate strong regression testing before anything goes live. Sound knowledge of the testing process and deep business-process knowledge are required to test the SAP application. SAP testing involves repeated checks of a specific functionality, and decentralized teams often lack the benefit of reusability perspective. This is another contributing factor for inflated cost.

A testing project team primarily looks at testing from a narrow point of view and often misses the holistic implication of change from a complex business perspective. This contributes to higher efforts in user acceptance testing (UAT) and a large number of defects identified in later stages. Non-functional testing aspects also get overlooked in the initial phase, leading to more re-work and higher costs.

4. Test Automation

Teams working in silos often lack a test automation-perspective. Most of the time, companies are apprehensive about the initial investment in automation and stick with traditional ways of testing. Even if a team invests in test automation, it often doesn’t realize the full benefits due to a narrow focus and lack of best practices, which perpetuates higher cycle times or reduced test coverage. Adding to that, SAP Automation comes with a unique challenge: automation engineers are expected to have sound business process knowledge. If an engineer lacks SAP business process knowledge, key areas of the system could be overlooked in testing jeopardizing the frequent change cycle.

The solution: Establish SAP Test Factory in Three Key Phases

Establishing a centralized Test Factory can be an effective solution for pharmaceutical companies that use SAP. Test Factory is a centralized model that brings people, process, and tools together in a shared service model. This model supports standardized processes, synergized and optimized resource usage, effective tool usage, higher re-use, and best practices that can elicit benefits from a cost, quality, and effort perspective.

SAP Test Factory can be operated in a model in which business value is measured against the services provided. For example, in a unit-based pricing model, cost is directly linked to deliverables from the Test Factory, whereas a traditional time-and-materials model spends a large amount of time (and thus cost) addressing unknown factors.

The SAP Test Factory implementation includes three phases:

1. Definition

This is the most critical phase and is where companies can assess the maturity of their existing processes – a required step. Here, teams can include regulatory and compliance aspects like GxP validation and SOX validation, the pain points of existing ways of working, and the pros and cons for each. Certain discussions can be setup in the form of a questionnaire or interviews with existing stakeholders (e.g. test director, operations director, and business stakeholders). Test managers should participate in discussions to explain the test-management process around requirements, test coverage, impact analysis, and the test-execution process. A good understanding of the organization’s regulatory and compliance process is a must, as any disruption or compromise to compliance can have negative impacts from a reputation, financial, and regulatory perspective. Assessment results aligned with leadership will set the tone for the solution phase.

2. Design 

Design is the core component of Test Factory setup and is based on the following layers:

a. The test delivery component focuses on core execution services like test execution, automation scripting, execution, and maintenance. The focus is to ensure that the delivery team should have a blend of knowledge across all SAP Modules. The delivery team should be aligned to a specific module of SAP like sales and distribution (SD) and material management (MM) as primary skills, and it have should have knowledge in other SAP modules or in test automation as a secondary skill.

b. The test engineering dimension of Test Factory focuses on setting up a test automation framework, impact analysis, test plan, etc. to ensure there is a defined framework for the test delivery team.

c. The test management and governance component works as an interface for testing and project stakeholders. This layer is responsible to ensure all defined standards, compliance, and test coverage are followed. This team is required to have good SAP knowledge to become an effective interface with the project team and validate all test deliverables and standards.

3. Implementation

Based on the assessment of an organization’s maturity and desired level of maturity, SAP Test Factory implementation may take somewhere between three months and two years. Organizations may choose an incremental or “big bang” approach for Test Factory, but it’s advisable to take an incremental approach to setup processes to manage changes effectively and without disruption. 

It may start with any of the following:

• Consolidating test management to align with a specific SAP functional tower
• Setting up a solid foundation of the test-automation framework for projects, regression, etc.
• Expanding test automation for projects
• Expanding testing services to functional testing aligned with specific SAP Modules

Benefits of SAP Test Factory Setup

Setting up Test Factory leads to qualitative and quantitative benefits for organizations.

Qualitative benefits may include a focus on end-to-end business process testing, a high level of repeatability and consistency, an automation-driven delivery, a high level of reuse, a standardized process framework, effective use of tools to drive optimized usage, a focus on testing best practices, and a one-stop shop for all testing requirements (e.g. regression, security, performance, test management, test automation, etc.).

Quantitative benefits may include an estimated cost savings of 30% by leveraging synergy across teams, a strong compliance framework driving near zero-defect delivery, an accelerated testing with automation.

Conclusion

Setting up the SAP Test Factory enables multiple qualitative and quantitative benefits. For pharmaceutical companies, regulation and compliance is critical. A company’s testing processes must ensure that all requirements are met and do so in a timely manner. Continuous SAP testing, leveraging the SAP Test Factory can be a key future differentiator for pharmaceutical companies looking to transform themselves into an intelligent enterprise designed for the ever-evolving digital reality.

Shyam Agrawal

Shyam Agrawal is Program Manager at Wipro with 15+ years of experience in leading and managing SAP Testing engagements at global clients. He has led global transformation programs focusing on building corporate competencies and providing strategic direction. He can be reached at shyam.agrawal1@wipro.com