by J. P. Russell
For management control purposes, there are far more audits of a product, service or process than audits of systems. While system audits verify management system conformance or compliance, product and process audits focus on the verification of specific methods and product or service characteristics.
Grouping product, service and process audits together is somewhat natural, because a process audit may include a product or service audit. I’ve dubbed the combination a verification audit.
Verification audits are part of what Joseph Juran called the “little q” (quality control, tactical tools) as opposed to the “big Q” (quality assurance or management systems). System thinking is important, but you can’t lose sight of the everyday tools necessary to ensure processes are controlled and risks are minimized.
Supply chain management, outsourcing, process or product complexity and sophistication, certified suppliers and operators, global economies and risk of field failures have all increased the need for ongoing verification. In addition, verification audits need to be performed when there are routine changes in suppliers, equipment, process settings, methods, requirements or personnel.
Meeting requirements
Standards require verification of products and activities to ensure control as part of the plan-do-check-act model.
For example, the ISO 13485 medical-device standard uses the words “verification” and “validation” more than 100 times.1 Most verifications and validations are integrated into design, manufacturing, or service-delivery processes and are performed by operators, inspectors, technicians, engineers, service providers, and auditors.
According to clause 7.4.3 of ISO 9001:2008, organizations must establish and implement activities to ensure purchased product meets specified requirements.2 If risks of nonconformity or failure are low, verification may be a simple inspection. If risks are high, however, supplier processes may be verified, contract requirements affirmed, and product or service characteristics and performance checked.
Risks could be high due to complexity; low confidence levels in the supplier; sole sourcing; criticality of the product or service; large amounts of revenue being transacted; material; or safety, health and environmental consequences. Considering the number of product recalls recently, contaminated foods and reports of defects, it seems these risks are greater than ever.
Know your source
The benefits and market advantages of outsourcing has increased reliance on other organizations for important products and services. The organizations that provide the outsourced products and services are run by managers with different goals, skills and values. Increased oversight is needed to ensure suppliers provide what they promise, now and in the future.
ISO 9001, clause 4.1, allows outsourcing of any process, but the organization must ensure control over any outsourced processes and retains responsibility for conformity to all customer, ISO 9001, statutory and regulatory requirements.3
Many organizations do a good job of verifying materials and components but don’t do as well when it comes to services or processors. Supplier services or processors may, in fact, have the greatest impact on an organization.
A global economy gives management more options to ensure the organization is effective, efficient and able to survive competitive pressure or increased demand. The advantages could be negated, however, due to failure costs, delays or risks to the wealth of the organization (assets).
Some organizations purchase sophisticated or specialized services that require special equipment or individuals with a particular expertise or trade. In other cases, organizations are becoming more reliant on second-party services to reduce overhead costs, free up internal resources and space, and procure services that aren’t their strength.
Organizations carry out projects to design, develop, construct, assemble or build. Designing may require seeking special expertise for various nuances external to the organization. For example, if you are designing a tower, you may need a wind expert. If you are constructing a building, you may need a foundation expert.
The performance of design, development, construction, assembly or building activities may require performance of tasks by suppliers that represent a high risk to the successful outcome of the project. This comes into play in design calculations, modeling, and quality of concrete and pour method, to name a few examples.
Also, there are high-risk activities performed by employees that need oversight. Due to consequences of failure, some risks must be mitigated by verification and validation.
Clause 7.5.2 of ISO 9001 requires organizations to validate any processes for production and service provisions in which the resulting output cannot be verified before delivery to the customer.4 For example, there may be sophisticated equipment that must be operated and calibrated (applicable in welding, testing pharmaceuticals, medical-device validation and containerization of dangerous materials).
Prepare to verify
One way to mitigate negative consequences of high-risk processes or product use is to conduct verification audits. Verification audits can be a product audit, a process audit or a combination of both.
Product and process audits are considered preventive actions—a form of quality assurance rather than inspection, which has been thought of as quality control.
Once it is determined an audit is needed, standard audit preparation activities should be carried out. You will need to know the audit objectives, criteria and scope. There could be several audit objectives depending on whether it is an internal or external audit, as well as the performance history of the process, product or service. Following are some of the objectives that should be achieved:
First-party audit objectives:
- Verify and validate a process. The process may be an activity that can’t be verified by inspection or test. The process may be a special test or procedure requiring special expertise or equipment.
- Verify project implementation activities, such as new products or services.
- Verify product characteristics or validate performance expectations.
- Verify that defects and nonconformities have been addressed.
- Verify training, equipment capabilities and process settings.
Second-party audit objectives:
- Verify supplier organization processes used to provide a product or service.
- Verify supplier product or service characteristics or performance requirements.
- Verify supplier process capabilities.
- Verify conformance to contract requirements.
- Verify material sources and traceability.
- Verify that defects and nonconformities have been addressed.
Third-party audit objectives:
- Approve or disapprove process for license or certification.
- Approve or disapprove product or service for license or certification.
Next, the scope needs to be established. The scope may be an internal or external process or product. There may be one process, processes in a series or parallel processes that need to be verified. There may be one product or service that needs to be verified, or there may be several.
You need to understand the process and product or service you are going to audit. Reviewing the procedure, specifications and records is a good starting point. If there is no procedure, you may need to ask the auditee to provide a description of the processes. Talk to people to get the information you need.
The right tool
There are several tools available that can help you understand the process. They include:
- Process flow diagrams, flowcharts or process mapping.
- Cause and effect, turtle and tree
- Failure mode effects analysis.
- Training documents.
- Inspection checklists.
- Bill of materials, quantities and specifications.
There may also be some type of process input and output criteria, including:
- Specifications and lists.
- Drawings, pictures and diagrams.
- Planned arrangements for process approval.
- Approved equipment and qualification or certification of personnel.
- Test procedures.
- Inspection method sheets.
- First article inspections.
- Contract or regulatory requirements.
Find out about process or product history, including:
- Nonconformance reports and trend analysis.
- Internal and field failures.
- Corrective actions.
- Process or product changes, date and nature of changes.
- Operator or technician changes.
- Revalidation history.
- Customer complaints.
The process elements to be considered for a verification audit are summarized in the simplified spider diagram in Figure 1. Such diagrams can be used to check off the areas verified and can be customized for your situation.
Performing the audit
Follow standard auditing protocols for conducting the verification audit. If it’s an internal audit, briefly make contact with the manager or supervisor before starting. If it’s an external audit, you will need to hold a short meeting with the manager or supervisor to review the audit plan.
If you are going to check product in addition to the process, you will need to determine your sampling method. For example, you may choose to observe sample selection, inspection and test procedures.
Primary strategies for process audits:
- Tracing and process strategies.5
- Test the weaknesses and verify the strengths of a process or series of processes.
- Collect information by using open-ended questions to gather data about process inputs, outputs and the process elements (people, environment, equipment, material, measuring and method).
Primary strategies for the product or service audit:
- Use the requirements.
- Use element/clause strategy.
This method is used in system audits to determine if an organization conforms to requirements specified in elements or clauses of a standard. This same strategy is used in product and service audits to determine if a product conforms to specified requirements that are in an element or clause of a standard, specification or condition document. A product audit can include verification of product characteristics, as well as performance requirements.
Verification audits are perfect opportunities to perform reverse traces, because there are more verifiable links throughout a process or processes. The traces can include, but are not limited to, process settings, personnel training, material traceability, nonconformance handling, material handling and storage, shelf-life controls, process equipment, measurement system and supplier control.
You can also go back to the start and check requirements such as purchase orders, contracts and regulatory or internal requirements.
Verification audits can include error-proofing or mistake-proofing to improve process effectiveness and efficiency if they’re part of the audit objectives and purpose. Identified weaknesses can be a potential source of nonconformity if not addressed. An auditor may also observe opportunities for improvement that can be reported if included in the audit purpose.
Reports are normally brief and address the audit objectives. Reports should describe the items reviewed—whether they are characteristics, processes or documents—and the audit results based on the requirements or expectations.
The report should also define the requirements for corrective action and preventive action when appropriate. You can follow up on audit results via records showing that findings have been addressed or via a subsequent audit.
Final thoughts
Verification audits are part of a robust risk-management process to mitigate potential unacceptable losses. The frequency of verification audits depends on degree of risk and performance history.
Processes and the environments in which they take place are constantly changing. Verification audits are key tools (“little q”) to ensure sustainability of the management system (“big Q”). Results come from checking, not expecting.6
References
- International Organization for Standardization, ISO 13485: 2008—Medical devices—Quality management systems—Requirements for regulatory purposes.
- International Organization for Standardization, ISO 9001:2008—Quality management systems—Requirements.
- Ibid.
- Ibid.
- P. Russell, ed., The Auditing Handbook, third edition, ASQ Quality Press, 2006, pp. 80-81.
- P. Russell, Continual Improvement Assessment Guide, ASQ Quality Press, 2003.
About the author
J.P. Russell is an ASQ Fellow and a voting member of the American National Standards Institute/ASQ Z1 committee. He is a member of the U.S. Technical Advisory Group to Technical Committee 176, the body responsible for the ISO 9000 standard series. Russell is the managing director of the internationally accredited QualityWBT Center for Education, www.jprlearning.com, an online auditing, standards, metrics, and quality tools training provider. A former RAB and IRCA lead auditor and an ASQ Certified Quality Auditor, Russell is author of several ASQ Quality Press bestselling books, including Process Auditing Techniques; Internal Auditing Basics; ISO Lesson Guide 2015: Pocket Guide to ISO 9001:2015; and he is the editor of the ASQ Auditing Handbook.
This article first appeared in the ASQ Quality Progress magazine November 2009. The information is relevant today.