The 7 most common risk management errors

This article identifies the seven most common risk management errors that Johner Institute and its auditors encounter most often. It also offers advice on how to avoid these errors.

Risk management is among the most important requirements medical device manufacturers must meet. Therefore, it is important that they avoid risk management errors.

1st error class: Incorrect use of terminology

ISO 14971 defines the relevant terms such as harm, hazard, hazardous situation, risk, and severity. Here, the manufacturers regularly make mistakes:

• A malfunction of a device is called a hazardous situation, although it would be a hazard.
• The "risk" column contains entries such as "electrical energy," although this is the hazard.
• The manufacturer lists radiation harm in the hazards, even though that is a harm.

There is also usually a lack of common understanding within the company as to which element of the chain of events should be entered in the hazard column.

It becomes particularly tricky when manufacturers calculate the risks as the product of probability and severity of possible harm. This makes no mathematical sense, nor does it conform to the definition of risk.

The risk priority number, i.e., a quantity of three factors, one of which is detectability, does not correspond to risk in the sense of ISO 14971.

2nd error class: No systematically derived acceptance criteria

The manufacturers must derive the risk acceptance criteria for each device. It is, therefore (almost always) an error in risk management to define the risk acceptance matrix globally in a SOP.

Manufacturers should also not estimate the risk acceptance criteria but derive them mathematically if possible. In doing so, they should avoid another error:

They should not have quantitatively and qualitatively demonstrated a different benefit in the clinical evaluation than is being used in weighing the benefit-risk ratio and, thus, in deriving the acceptance criteria.

A prerequisite for manufacturers to express acceptance criteria in the form of a risk-acceptance matrix is that the axes be precisely defined. Popular errors in this regard are:

• The events in the probability class "unimaginable" occur mathematically hundreds or even thousands of times in all applications for all devices over the complete lifetime and are thus anything but "unimaginable."
• The classes quantifying the severity of harm are not precisely defined using binary decision characteristics.

3rd error class: Incomplete hazards and risks

The next type of risk management error concerns the completeness of identified hazards and, thus, risks. The reasons for completeness are manifold:

• The manufacturer has not applied the hazard analysis methods accurately or at all.
• The failure consequences he identified with the design FMEA or process FMEA do not appear as hazards in the "risk table."
• The risks due to lack of usability have not been systematically identified, for example, because usability tests have not been carried out or have not been carried out with representative users and not in representative use environments.
• The manufacturer does not consider the hazards before the risk mitigation measures , e.g., before choosing a two-channel design. This would not be compliant and would not ensure that for these hazards, the measures are still tested for effectiveness.

4th error class: Risks incorrectly assessed

The next risk management error is that manufacturers misjudge the probabilities and severities of harm. This, in turn, has several causes:

• Estimation is difficult per se.
• The manufacturer has not considered the entire chain of events. He, therefore, lacks the information necessary for the estimations.
• The staff does not have the necessary competence. For example, only a medical doctor can estimate the probabilities and severities that arise from a given hazardous situation.

Many manufacturers tend to investigate only the worst case (e.g., death). However, it is a mistake to assume that the greatest risk always occurs at the greatest severities of harm.

5th error class: Risks from the post-production phase overlooked

However, even with the best method and the greatest experts, the errors in risk management files mentioned in sections three and four cannot be avoided.

Therefore, it is essential to continuously collect information in the post-production phase, in particular as part of the post-market surveillance, to complement and improve the risk analysis.

Another reason is the changing state of the art, which the risk acceptance criteria must reflect.

The post-production phase, as defined by ISO 14971, also includes production. Regularly, manufacturers forget to analyze the risks caused by production. Whenever production is changed, manufacturers must update this analysis. However, this is exactly what is often omitted, which means another error in risk management.

6th error class: Insufficient validation of the measures

Another class of errors concerns the validation of measures. Either this validation is omitted altogether for individual measures. Or the manufacturers are not aware that two validations are necessary:

1. Verification that the measure has been implemented, e.g., through a review of the device design.

2. Validation that the measures are effective, typically through appropriate tests.

There is a tendency not to establish inherently safe measures or safeguards but to determine accompanying materials and training as means of risk control. In doing so, manufacturers should avoid two other risk management errors:

1. All measures, including accompanying materials, must be verified respectively validated by manufacturers - in this case, through summative evaluation.

2. There is information that must not be used as risk mitigation measures.

7th error class: Formal errors

The seventh and final type of risk management error relates to formal requirements that manufacturers fail to meet:

• They fail to review and release documents in the risk management file.
• They lack specifications (e.g., in the form of an SOP) that require exactly this.
• The creation and release date are illogical. For example, the risk management file is released even though changes are made to the device afterward.
• It is no longer possible to show which version of the product refers to which version of the documents because both the devices and the documents are continuously being developed.
• Necessary documents are missing from the risk management file, such as the risk management plan or the risk management report.
• The documents are missing necessary contents, e.g., for document control (like author, version, date, status) or for conformity. For example, the report lacks a statement on whether the risks are acceptable given the benefits.

Summary and conclusion

None of the steps in risk management are simple:

• Establish a risk policy and risk acceptance criteria
• Create a risk management plan
• Identify the hazards
• Evaluate the risks
• Determine and verify measures
• Writing the risk management report

Nevertheless, typical errors in risk management can be avoided. To help with this:

• The tips mentioned in this and the linked articles
• An interdisciplinary and competent risk management team
• A management that understands the importance of risk management and provides this team with the necessary resources
• The seminar "Risk Management and ISO 14971", not only provides a quick introduction to the topic but also allows the activities to be practiced