Autonomous Systems: 4 Advantages for Medical Device Manufacturers

For those of you who want to learn more about these systems, we recommend reading Autonomous Systems in Anesthesia: Where Do We Stand in 2020? A Narrative Review and Robotic anesthesia: not the realm of science fiction any more.

b) Definition

There are also other definitions because “autonomous” really only means “independent” or “self-contained.” The definitions always reflect what is important for a particular system, for example:

• Independent recharging by mobile robots
• Flying without detectable remote-control signals for military drones

A fundamental characteristic of autonomous systems is their ability to adapt, i.e. to adapt to different situations by themselves.

c) Difference from automation

In the case of automation, the main aim is replacement, with autonomy it is independence. You “automate something” but are “independent of something.” Automated driving indicates that the “driving” has been automated. Autonomous vehicle indicates that the vehicle is independent of the driver.

However, in general, autonomous systems are able to work independently even in complex and changing environments. This is not the case with standard full automation:

an industrial robot that has been permanently programmed to place a component on a conveyor belt acts completely automatically but is still not an autonomous system according to the above definition.

In contrast, an industrial robot that independently searches for components and assembles them on cars, even if there are different models, the cars are positioned differently or people are in the robot's working area, is an autonomous system.

d) Difference from closed loop systems

A lot of autonomous systems include closed loop systems. The popular terms “Sense Understand Decide Act” (SUDA), “Monitor Analyze Plan Act – Knowledge” (MAPE-K) and “Cognitive Loop” are all aimed at the following steps of the control loop:

• Perceiving the situation
• Making a prediction
• Making a decision
• Implementing the decision

Unlike in “classical” closed loop systems, in autonomous systems the rules for the prediction and decision-making process are often not so rigidly or explicitly formulated, i.e. programmed. Machine learning methods are regularly used.

In addition, autonomous systems are usually also networked systems, because distributed sensor technology is often an advantage when it comes to gaining a sufficient understanding of the situation. This is not necessarily the case with closed loop systems.

2. Specific risks from autonomous systems

a) Risks resulting from the system's autonomy

Autonomous systems work independently of, for example, human intervention. This often means that the opportunity for human intervention is lost, which increases the probability that a system error will remain undetected and lead to harm.

b) Risks resulting from different technical contexts

These systems are adaptive. They are designed to be able to adapt to different contexts. The contexts will be different in many aspects, including technical aspects, such as the:

• Quality (bandwidth, latency, availability) and type (different protocols, wired versus wireless) of network connection
• Type of devices that are part of the system (e.g., different models from different manufacturers, differently configured devices from one manufacturer)
• Number of devices that are part of the autonomous system. This number may change as the operator adds or removes devices
• Devices may also fail, stop being available or no longer offer the promised performance as a result of technical problems in the network or the device itself
• Physical environment (brightness, humidity, noise, dirt, etc.)

c) Risks resulting from different clinical contexts

The autonomous systems also have to behave safely in different clinical contexts. Attributes of these clinical contexts include the:

• Number, availability and training of staff
• Mental workload, stress, e.g. from alarms, shift operation, shift changes, emergencies
• Number and state of health of the patients
• Availability of resources, medical devices and other systems

d) Risks from adaptive algorithms

A lot of autonomous systems acquire the ability to adapt to different contexts by using adaptive algorithms, e.g., artificial intelligence.

e) Risks due to lack of interoperability

The more (medical) devices connected to an (autonomous) system, the disproportionately higher the number of interfaces that have to work together is. There are often problems at the semantic interoperability level. That's why the FDA is pushing for the use of international standards.

To be able to make these context-dependent decisions, all the necessary information on the individual devices and the context must be available.

Interfaces, models and domain-specific languages

This, in turn, requires a standardized exchange of data between devices and manufacturers and, therefore, common data models.

Domain-specific languages (DSL) describe precisely these data models and, as a result, the interfaces.

The individual devices that are part of an autonomous system don’t just communicate their values (e.g., sensor data) and instructions via these interfaces, they also give information about themselves, such as their internal status and the reliability or uncertainty of the values communicated.

The Fraunhofer IESE, for example, has developed DSLs and corresponding tools (including for MagicDraw and Enterprise Architect). The description language used to describe reliability is based on a standard from the Object Management Group (OMG) and has been published as open source.

Decision-making unit

If all (safety-relevant) information is available from all devices and for the relevant context, a (usually central) decision-making unit is required. This unit:

• Checks whether the assumptions regarding the medical context have been fulfilled
• Checks how completely the technical requirements (that the manufacturer assumed at the time of development) have been met
• Anticipates the context-specific consequences of an action (e.g., administration of a drug through an infusion pump when there is uncertainty about one of the devices)
• Executes the decisions and controls devices, e.g., an infusion pump or an alarm

This decision logic should also be formulated in domain-specific languages. This enables interoperability across devices and manufacturers.

A high degree of domain knowledge is required to describe this “business logic.” Ultimately, this leads to a formalization of medical knowledge.

Summary

It is no longer just individual devices that make decisions for themselves, autonomous systems (or their decision-making units) now make decisions on the behavior of the entire system. The system makes these decisions independently and context-specifically.

Domain-specific languages describe both the context and the decision logic and as a result:

• The devices in the system
• Their interfaces
• The effects of decisions in the event that the assumptions are fulfilled and in the event that they are not (e.g., devices do not behave according to the specifications)

4. Advantages of autonomous systems

a) Four advantages for medical device manufacturers

Higher patient safety

Manufacturers can and indeed must define the intended purpose and intended use and, therefore, have to establish the clinical and technical context.

However, they are hardly in a position to assess the risks that could arise with every possible combination of devices and contexts. This applies in particular to situations where users do not use the devices as intended.

As a result, it is difficult to manage these risks and ensure patient safety.

Autonomous systems can (and should) check whether the assumptions made by manufacturers are fulfilled at any given time and then make a context-dependent decision on how safety can also be guaranteed even if the manufacturer’s assumptions are not fulfilled.

No unnecessary device costs

A lot of manufacturers try to estimate the risks for the combinatorial range of situations by making worst-case assumptions. But these worst-case assumptions regularly require unnecessarily expensive risk-minimizing measures. These lead to higher costs or even to the device no longer being competitive and therefore no longer being placed on the market.

Autonomous systems should not start from a hypothetical worst-case scenario but make decisions based on actual cases. Systems have more options available to them for reacting to a specific context than individual devices.

As a result, system manufacturers are not just able to implement more economic (more efficient) risk minimizing measures but, using dynamic risk management, they are also able to implement more effective risk minimizing measures. This, in turn, leads to higher patient safety.

More efficient development, documentation and authorization

Most manufacturers try to meet their customers’ requirements with device variants and device families while limiting development costs.

However, they cannot do this cost-efficiency if they have to create separate technical documentation for each device and each variant and then have to go through a separate authorization process.

These apparent dilemmas are where model-based development comes into its own:

• Safety strategies only have to be modeled, developed and tested once. Modeling ensures that these strategies have been implemented identically and effectively in the different variants and devices. Points of variation in the device line's engineering artifacts are connected to the points of variation in the safety artifacts and create a safe line of devices.
• Modeling, e.g., using domain-specific languages or modeling tools, leads directly to the interface specification (and thus to device or component requirements). The models also form part of the consistent documentation, i.e. they are part of the technical documentation.
• This also increases “regulatory compliance” because the frequently encountered and usually incomplete “post-documentation” does not happen with these model-based approaches.
• In addition, models allow verification and validation of the device or component at the model level.

Conclusion: Modeling devices and safety strategies in particular can save manufacturers development and documentation time and costs. This advantage increases the more often devices or components are reused in different combinations, e.g., in device families.

Increasing the efficiency of other processes

This article only looks at autonomous systems in the context of medical devices. But manufacturers can also use autonomous systems to produce medical devices (comparable to the car industry) and in the automation of research (see a press release from the Fraunhofer IESE).

b) Three advantages for operators (hospitals, clinics, practices)

Legal compliance

Laws and regulations, such as the Medizinprodukte-Betreiberverordnung, establish special requirements for healthcare facilities that use networked medical devices:

But how should operators meet these requirements? How can they be able to assess the risks if the only information they have about the devices is what is contained in the accompanying material?

Autonomous systems with device interfaces are described in domain-specific languages help to ensure that the operator can comply with these requirements.

Currently, these requirements are only detailed in the accompanying material (if at all). But these interface descriptions, which can also be evaluated by computer, are a necessary prerequisite for the use of the device.

Informed selection of devices

They allow healthcare institutions to make an informed choice of devices: only devices for which these interface descriptions are available are considered for use in autonomous systems.

Just as DICOM declarations of conformity (with IOD, SCP, SCU) ensure the interoperability of DICOM-capable devices, devices with the interfaces described using DSLs make it possible to select interoperable devices more precisely.

Operators can explicitly specify these interoperability requirements in requests for proposals. It is also easier to select devices and replace them with other devices if they have identical interfaces.

More safety for patients

However, the most important advantage for operators is that the interoperability of the devices and the ability of autonomous systems to respond specifically to different contexts will lead to increased patient safety.

Now patient safety doesn’t depend on the timely intervention of medical staff and the specific medical and technical context because autonomous systems are characterized by the fact that they are autonomous and can adapt to different conditions.

5. Autonomous system from the perspective of medical device law

a) Medical device versus system

The term “system” brings to mind systems consisting of several devices, e.g. in the sense of “systems and procedure packs” according to Article 22 of MDR. But this is not always the case.

Autonomous systems can be:

1. An individual medical device, such as a surgical robot. This would be a PEMS (programmable electrical medical system) according to IEC 60601-1.
2. A combination of devices placed on the market by a company consisting of at least one medical device and, if applicable, other devices or products as defined by article 22.
3. Several products or devices (including medical devices) connected to one another by a healthcare institution (e.g., hospital) that are together intended to achieve a specific purpose.

The MDR and IVDR only establish specific requirements in the first of these cases. The risk management requirements are particularly challenging. This is because manufacturers must assess the combinatorial explosion of devices, the statuses of devices the device is used with and contexts the device is used in.

Standards such as IEC 80001-1 describe the risk management processes that healthcare institutions who combine devices into a “system” should follow.

b) Interoperability requirements

The intended use of an individual device may include being networked. Therefore, the MDR requires that manufacturers at least guarantee the interoperability of their devices. These requirements can be found, for example, in Annex I.

c) Classification

If a device contains a closed loop control system, manufacturers should study Rule 22 of Annex VIII of the MDR.

However, this rule applies to individual devices and the MDR does not contain any specific rules for systems consisting of several (medical) devices.

d) Conclusion

A lot of regulations, e.g., those in the MDR, do not take networked (autonomous) systems into account. They establish requirements that the individual devices must meet, including the above interoperability requirements.

The MDR does not regulate the responsibility for the functioning of an (autonomous) system consisting of several devices from different manufacturers. This responsibility lies with the operators. However, they are often overwhelmed by this very requirement.

6. Summary

a) Autonomous systems bring together a lot of challenges

The advantages of autonomous systems are obvious. But so are a lot of the challenges:

• These are networked systems made up of a lot of devices. The combinatorial explosion of configurations and problems is difficult to control.
• These devices often use artificial intelligence methods, and it is particularly difficult to demonstrate the safety and performance of these algorithms.
• The autonomous systems are generally closed loop systems that are used in critical contexts. The risks are particularly high in this context.

b) But there is no way around it

In the long run, it is naive to want to treat patients individually, safely and effectively with isolated medical devices and in ignorance of concrete technical and medical contexts.

The increasing shortage of medical personnel requires systems that actually react autonomously, i.e. without human intervention, to contexts such as the failure of system components.

c) Is there a shortage of regulations or are autonomous systems poorly regulated?

The EU regulations are not suited to the current situation. They focus too much on the individual device and its interoperability. The idea of a system – and not a system as defined by Article 22 – is missing.

Any manufacturers delighted that something has not been regulated may be celebrating too soon because a lack of regulation leads to unnecessarily high levels of uncertainty during the authorization of individual devices. The high demands from authorities and notified bodies and the discussion of unrealistic worst-case scenarios complicate the authorization process.

The belief that operators would effectively analyze and manage the risks of networked systems is no more than a belief.

d) There are big opportunities for good manufacturers

Manufacturers who already fail to make their medical devices safe by using modular architectures and who do not prepare modular documentation will find it difficult to develop devices for use in autonomous systems.

That's because modular architectures are a prerequisite for

• Reusability, interchangeability and testability of components in different devices and (autonomous) systems
• Effective safety strategies
• Extension with modular “safety artifacts”
• Lean, consistent and legally compliant documentation

In contrast, manufacturers who have already prepared model-driven modular architectures are well prepared. They will be rewarded with

• A quicker time to market
• Lower development and device costs
• Leaner documentation for their devices and device families
• An easier path to achieving and demonstrating legal conformity and, therefore, faster authorization
• Safer devices
• Greater competitiveness

e) Taking the first steps

Even the longest journey starts with the first step. Examples of first steps could be:

• Defining the clinical contexts that you, as a manufacturer, want your devices to be used in
• Reviewing the modularity of your own devices
• Determining the status of modeling in your own company
• Offering developers further training on the topics of:
  • Autonomous systems
  • Modeling
  • Domain-specific topics
  • Safety architectures
  • Risk management
  • Regulatory requirements
• Discussing a small feasibility study with experts in autonomous systems

As with the use of computer-based modeling and simulation, the hurdles for the development of autonomous systems are high. The development of autonomous system will not be successful without a clear strategy and long-term investment in staff, in strategies and tools.

In contrast, companies that commit to this path will benefit from vital competitive advantages.

Dr. Rasmus Adler from the Fraunhofer IESE and the Johner Institut team are available to answer any questions you may have, e.g., via the free micro-consulting service.