Software verification and validation activities are required by FDA, but they can also bring significant business value to an organization.
Bryan Chojnowski Reglera Corp.
For a variety of reasons, software verification and validation has proven to be one of the more challenging and nebulous areas of compliance for companies regulated by FDA. Software, and how it is used to automate part of the quality system or another business process, has evolved significantly over the past 10 years. Although some companies still develop custom software to satisfy a particular business need, many companies purchase off-the-shelf software that simply needs to be configured, or they may choose to customize off-the-shelf software.
Because most companies use different applications of varying complexity to meet business and compliance requirements, it can be challenging to determine what software needs to be validated. Moreover, it can be challenging to determine specifically how to validate each software system. This situation is compounded by the fact that the topic of validation is often discussed after the start of a software development or implementation project. Such delays create timeline, resource, and compliance challenges.
As software failures can affect business operations, production, compliance, the end device, the user, and potentially the patient, risk management should be an integral component of any software validation. FDA covers risks as they relate to device software, but these general concepts should also be applied to nondevice software.
By answering some basic questions about software validation, this article examines the critical concepts for successful validation.
What Are Some of the Key Attributes of Software?
Because of its relatively intangible nature, software may seem more difficult to comprehend and describe than, say, a piece of equipment such as an autoclave. Equipment function is often readily observable and therefore fairly easy to comprehend and describe. That said, from a verification and validation perspective, software does, however, have desirable qualities when compared with equipment.
Software Can Be Customized and Configured to Very Closely Match Business Needs and Processes. Software basically automates operations, so the better it fits a business’s operational needs, the more cost savings will be realized for the related process. Software is much more effective than humans at routinely performing data monitoring and complex calculations. If it is leveraged properly, a work multiplier
results for the related process. For example, software that automates the corrective and preventive action (CAPA) process can review feeder data sources and discern trends more quickly and accurately than a manual data review process performed by a person. The cost of performing the CAPA process has been reduced significantly because companies have leveraged well-suited software.
Given a Specific Input, Software Will Always Produce the Same Specific Output. Achieving this output, of course, assumes that the system configuration is controlled. In contrast, misadjustment or wear and tear can cause electromechanical equipment to produce a different output given the same input. This characteristic of software obviates the need for conducting multiple iterations to test the same functionality, and thus reduces the sample size to a value of one.
Software Gets Better With Age. A standard life-cycle process includes feedback loops, which allow software issues to be addressed and functional improvements to be made. The result is that the robustness and usefulness of the software system improves over time.1 Equipment, in contrast, usually wears out over time and needs to be replaced.
Software Functionality Can Be Compartmentalized. This function allows individual software modules to be independently verified during development, prior to completion of the entire system. Equipment often needs to be fully assembled before functionality can be tested, which increases project risk because problems are then identified late in the process.
These characteristics can prove advantageous to the process of software verification and validation. In many ways, software can be more effectively defined and validated than equipment.
Why Should Software Verification and Validation Be Performed?
Figure 1. (click to enlarge) Potential benefits of software validation and verification.
Software verification and validation can reduce business costs and improve software performance. The core concept of validation is to prove that a system meets the requirements for its intended use. Done effectively, verification and validation accomplish more than compliance because they affect the cost of system ownership and, as such, are good for business (see Figure 1). This result is accomplished through two primary verification and validation outcomes: defect reduction and improved focus on the key purposes of the system.
First, costs are incurred each time a defect is addressed or a functional enhancement is implemented after the system is released to production.
The cost of development, testing, documentation updates, and validation can be high for even the smallest of software changes. A defect could also result in significant system unavailability, which could have widespread negative business ramifications.
Second, and often overlooked, is that improved focus on the key purposes of the system significantly reduces the time required for resources to perform a particular business process. As noted above, properly designed and implemented software can serve to multiply and improve the quality of the output of resources allocated to a particular business process. A well-thought-out and implemented verification and validation plan is the vehicle to ensure the system is properly designed and implemented. Properly executed verification and validation activities affect both the areas described above in three ways:
• They reduce the likelihood that a critical defect will be found in the primary utilization path of the software.
• They ensure that user needs and intended uses are satisfied by the software system, so that the need for future system enhancements is reduced.
• They allow system refinement to occur so that process optimization is approached and software functionality is leveraged as a work multiplier.
If you take into account that ongoing maintenance and use of a software system can account for 80–90% of its total cost of ownership, effective performance of verification and validation can significantly affect a company’s bottom line.
Verification activities such as technical reviews are an excellent way to ensure that a system’s design or configuration truly meets the intent of the requirements. There is often a gap between the individual who develops or configures a system and the individual who knows how the system should work. One has the business knowledge, whereas the other has the technical knowledge. Discussions between these individuals help achieve the best technological solution to each particular business requirement. Communication also allows flaws to be detected and fixed before the software moves to production use.
Validating that a piece of software meets user needs and intended uses ensures compliance with applicable regulations but also can ensure that a company’s critical business needs are met. Be sure to receive credit for all formal and informal verification and validation activities by documenting each for inclusion in the project documentation set.
What Software Needs to Be Validated?
Software used for compliance with predicate rule (21 CFR Part 820, 21 CFR Part 1271, etc.) must be validated. In some cases, only portions of a software application are used for compliance with predicate rule. An example is an enterprise resource planning (ERP)
application.
Systems such as ERPs may provide functionality necessary to comply with predicate rule (e.g., nonconformances, labeling, or distribution) as well as functionality unrelated to FDA compliance (e.g., material costing, human resource time tracking, or production planning).
In the case of an ERP system, validation activities should target FDA-compliant functionality. As noted, it is advantageous from a business perspective to validate most of the system functionality. It improves focus on the key purposes of the system, and thus creates potential work multipliers.
What Does Software Validation Entail?
At its most rudimentary level, software validation involves the execution of tests designed to cover each of the specific system requirements. System requirements should be written to be unique, unambiguous, and discretely testable. Requirements can be created from scratch for a new system or derived from existing documentation for an off-the-shelf system. Either way, it is recommended that a company document its own specific requirements for a software system. In the case of off-the-shelf software, this approach prevents a company from relying too heavily on the software vendor’s interpretation of how the system will be used. Instead, documenting requirements will capture how the software system will be used to satisfy the company’s specific
business needs.
When Should I Start Software Verification and Validation Activities?
Verification and validation activities should begin in the project planning phases. Software validation is not something to be left to the end of the project. Software requirements definition and verification and validation generally each take about 20% of the overall project time for a custom system. These percentages only increase for systems that require less software development work.
An overall project plan should be created to lend visibility to each of the major project activities, resources involved, and milestones so that supporting activities such as verification, validation, user documentation, and training are given proper emphasis. If a particular business objective is to be achieved through the implementation of the software system (e.g., reduce complaint throughput time by 25%), consider tailoring verification and validation activities to help shape the system to achieve the desired outcome.
How Should Software Requirements Be Organized?
It is often beneficial to organize software requirements by functional role. This makes logical sense because users of the system will likely be performing activities related to a particular business role. Organizing the requirements in this fashion helps ensure full functional coverage, improves the focus on key purposes of the system, and contributes to efficient construction of test protocols. As an example, a complaint-handling system typically has several different users, including
• A system administrator to set up users, assist with configuration, provide security, etc.
• A complaint coordinator designated to field complaints and to enter initial information.
• A complaint investigator to review each complaint and determine its status, conduct a root cause investigation, document findings, follow up with the customer, etc.
• A regulatory reviewer to assess FDA MDR compliance, vigilance reporting status, and so on.
• Complaint approvers to ensure that the complaint information is complete prior to closure.
In addition to requirements associated with the user roles, there are often requirements that describe general or broad system functionality, such as category trees, entity (e.g., company, contact) maintenance, audit trailing, electronic signatures, system security, and so forth.
Requirements for the system should be an early project deliverable that will be refined as the software reaches its final configuration. The requirements should help facilitate preliminary and ongoing testing of the system in preparation for validation.
What Approach Should Be Taken when Writing Validation Tests?
Complex software nearly always has bugs. Because it is not feasible to test every possible use pattern for every module of code, it is important to ensure that critical and high-risk functionality is defect free. As each system requirement is solidified, a corresponding test or set of tests must be generated to ensure that the requirement is met.
Although there may be several ways to test a particular area of functionality, tests should ensure, at a minimum, that the most common use of the functionality is exercised. In other words, be sure to test the primary intended use of the system before testing less-likely usage scenarios.
An 80/20 rule applies here: It is likely that you can test 80% of the primary intended functionality during 20% of the testing effort. The key point here is to ensure that important business and compliance requirements are satisfied before going through gyrations to test less-likely use patterns in the system. The underlying business advantage is that any defect found in production should be outside the primary use path; therefore, the software will always be able to accomplish its intended function by reverting to the primary use path until the defect can be addressed. While both common and less-common paths should be tested, be sure to prioritize appropriately.
What Supporting Processes Should Be in Place during the Project?
A process should be in place to identify, capture, and track project issues, software defects, and software enhancement requests. Issues will
arise during the course of any project, so a system should exist to track these issues through to resolution. The feedback loop created by such a system is critical to adjusting the course of a project and prioritizing and delegating activities to be completed.
Configuration management is the linchpin in any software implementation process. Configuration control allows an organization to properly manage each of the pieces of software and hardware that comprise a system deployment as well as the supporting validation, user, and system documentation. Failure to properly control software, hardware, and documentation can have dire consequences and cost a company significant time and energy.
Software typically runs in a system composed of other supporting software (e.g., operating systems, browsers, print drivers, network protocols, etc.) and hardware (e.g., server, personal computer, printer, peripherals, etc.). The specific requirements of each component of the system should be captured and controlled so that the system functions as intended.
What Additional System Documentation Is Required?
A traceability matrix should be created to demonstrate that each requirement is tested in the validation protocol. In addition, documentation needs to be in place to describe specifically how the system is to be used and maintained. User documentation can take the form of a set of procedures or a manual. Procedures work well in that they can be tailored to specific business roles and serve to complement more-generic system user manuals. Procedures covering system maintenance and data backup and recovery round out the system documentation set.
Conclusion
Software has many characteristics that make it easier to validate than equipment and that allow verification and validation activities to provide significant business value. Verification and validation can reduce the total cost of ownership of a system through defect reduction and improved work efficiency. Verification and validation should be incorporated into a project plan and should commence as early as possible during the execution of a software implementation project.
Think about organizing software requirements by business function to make them more usable and to ensure that they aid in a logical test progression. Supporting processes to track issues and manage configurations must be in place for any software project to be a success. Software projects have both incredible business potential as well as fairly significant downsides if not executed effectively.
Bryan Chojnowski is director of quality at Reglera Corp., an organization specializing in quality and regulatory consulting and process outsourcing. He can be reached at chojnowskib@reglera.com.
________________________________________
No comments:
Post a Comment