Neither the EU directives and regulations (e.g. MDD, MDR) nor the harmonised standards (e.g. EN IEC 62304) make concrete demands on medical devices which use artificial intelligence processes and machine learning in particular. For this reason, there is some uncertainty among manufacturers and notified bodies and it is often impossible to plan audits and licensing procedures. The Johner Institute can help you to get through both with confidence.

1. Advice and support from the Johner Institute

The Johner Institute supports you as a manufacturer of products based on machine learning processes throughout the product life cycle.

a) During the development stage and licensing

We help you with the following regulatory tasks:

• extending SOPs (specifically for AI applications), drawing up checklists
• formulating team competence requirements (requirements of ISO 13485:2016)
• clarifying system validation requirements (CSV) for gathering and processing test and training data; validating these systems
• formulating requirements regarding the efficiency, safety and robustness of medical devices and their machine learning processes respectively
• establishing interpretability and explainability requirements and selecting XAI (explainable artificial intelligence) methods
• determining procedures to verify and validate these requirements
• identifying risks specifically through AI processes, assessing them and developing risk control measures
• establishing the regulatory strategy and answering regulatory questions, e.g. if/when a continuous learning system requires a new licence, what tests are necessary in transfer learning and how to deal with SOUP updates
• assessing and selecting technologies and frameworks

The Johner Institute is also happy to help with questions regarding statistics and machine learning:

• assessing whether data are sufficient in terms of quantity, and are complete and representative
• assessing whether the data collection strategy is appropriate
• establishing the ground truth
• answering questions regarding training and the assessment of models, e.g. on the number and split ratio, on assessment of the model based on all data and on cross-validation
• selecting models (and their combinations) for the given problem
• transferring the model over to the product with other programming language, hardware and frameworks if necessary
• selecting methods to interpret the black box (transparency)
• solving problems, e.g. if the models do not depict the data (underfitting, lack of convergence)

b) After licensing and placing on the market (post-market phase)

The manufacturer’s responsibility does not end with the licensing of the devices. For example, they must carry out the following tasks:

• formulate procedure specifications on post-market surveillance specifically for AI devices; specify data that must be gathered and assessed, for example, to demonstrate that the actual population corresponds to that anticipated
• check whether performance is “degraded” by new input data, for instance (changed population, modified processes, e.g. in the case of imaging); identify reasons for a deterioration in performance and resolve the problem
• evaluate incidents, e.g. whether they are within the scope of the predicted probability; find out the causes, e.g. through a post-mortem analysis of the model
• proactively and continually gather and assess data
• monitor and assess regulatory changes relating to artificial intelligence and derive recommendations for action

The Johner Institute is also your specialist partner for these tasks.

2. For investors

Specially for investors, we answer questions within the scope of due diligence such as:

• How likely is the device to reach the promised targets, particularly with regard to clinical use?
• How safe is the device for patients, users and third parties? What are the risks and how well-controlled are they?
• How solidly has the manufacturer developed, trained and tested the software and the models respectively?
• How legally compliant are the product and its development? How likely are the product and the manufacturer to pass the audit and licensing procedure?

3. Training and further training

The experts from the Johner Institute support manufacturers, notified bodies and authorities through targeted training on artificial intelligence, e.g. in the form of:

• in-house seminars;
• webinars;
• video training in Auditgarant (from Q2 2019);
• individual coaching; and
• project supervision.

The subjects and focal points are:

• artificial intelligence processes;
• statistics and data assessment;
• explainable AI; and
• regulatory requirements.

Contact us right away without obligation. We would be happy to help you ensure that your medical devices contribute to better diagnosis and therapy of patients thanks to artificial intelligence and that they pass the approval procedures safely!