• Close menu
  • Home
  • Consulting
    • Technical Documentation
      • Software (IEC 62304, FDA)
      • Risk Management (ISO 14971)
      • Clinical Evaluation
    • Quality Management (QM)
      • QM System (ISO 13485)
      • Mock Audits & Inspections
      • Quality Representative
    • Services
      • Human Factors Engineering
      • Regulatory Affairs
  • E-Learning
    • Training videos
      • Market your medical device
      • Content
      • Premium Version
      • Watch training videos
  • Training Seminars
    • Medical Device Regulation
    • CPMS Seminar
    • Medical Device Software
    • Human Factors Engineering
    • Risk Management Seminar
    • Inhouse Seminars
    • Medical Device Regulatory Affairs
  • Articles
    • Regulatory Affairs
      • Medical Device Regulation
      • IVD Regulation (IVDR)
      • Medical Device Directive
      • Classification
      • CE Marking
      • Conformity Assessment
      • Harmonized Standards
      • MEDDEV 2.7/1
      • FDA
        • Breakthrough Devices
      • And more ...
        • Accessories
        • Essential Requirements
        • EUDAMED
        • MDR Rule 11 (Software)
        • MEDDEV Documents
        • Post-Market Surveillance
        • Notified Bodies
        • Premarket Approval (510k)
        • Technical File versus DHF
        • Unique Device Identification
        • Harmonized Standards
        • GDPR
        • 21 CFR Part 11
        • Design input
        • Declaration of Conformity
        • Systems & Procedure Packs
        • Laboratory Developed Tests
        • Free Sales Certificates
        • Physiological processes
    • QM System & ISO 13485
      • ISO 13485:2016
      • Quality Policy & Objectives
      • Process Validation
      • Audits
      • Unannounced Audit
      • MDSAP
      • CSV
    • Software & IEC 62304
      • Software Lifecycle
        • Agile Software Development
        • Software Requirements
        • Software Architecture
        • Code Reviews
        • Coding Guidelines
        • Software Testing
      • Safety Classes & Level of Concern
      • SOUP and OTS
      • Medical Apps
      • IT-Security
      • Artificial Intelligence
      • And more...
        • Parameterization
    • Artificial Intelligence in Medicine
    • Risk Management & ISO 14971
      • Life Cycle Risk Management
      • Update 14971:2012
      • Harm and Severity
      • Hazard and Hazardous Situation
      • Risk Acceptance
      • Risk Mitigation
      • Risk Analysis
      • Software Specifics
    • Usability & IEC 62366
      • Main Operating Functions
      • ISO 9241
      • User stories
      • Usability Validation
    • Product Development
      • Intended Use Description
      • System Architecture
      • Verification versus Validation
      • Clinical Evaluation
  • Login
  • Logout
  • Contact Us
  • About us
Search
+1 (301) 244-6335

info@johner-institute.com
  • LoginI
  • Contact UsI
  • About us
Search
Logo Johner Institut
  • Display menu
  • Close menu
  • Home
  • Consulting
    Technical Documentation
    • Software (IEC 62304, FDA)
    • Risk Management (ISO 14971)
    • Clinical Evaluation
    Quality Management (QM)
    • QM System (ISO 13485)
    • Mock Audits & Inspections
    • Quality Representative
    Services
    • Human Factors Engineering
    • Regulatory Affairs
  • E-Learning
    Training videos
    • Market your medical device
    • Content
    • Premium Version
    • Watch training videos
  • Training Seminars
    Medical Device Regulation
    CPMS Seminar
    Medical Device Software
    Human Factors Engineering
    Risk Management Seminar
    Inhouse Seminars
    Medical Device Regulatory Affairs
  • Articles
    Regulatory Affairs
    • Medical Device Regulation
    • IVD Regulation (IVDR)
    • Medical Device Directive
    • Classification
    • CE Marking
    • Conformity Assessment
    • Harmonized Standards
    • MEDDEV 2.7/1
    • FDA
    • And more ...
    QM System & ISO 13485
    • ISO 13485:2016
    • Quality Policy & Objectives
    • Process Validation
    • Audits
    • Unannounced Audit
    • MDSAP
    • CSV
    Software & IEC 62304
    • Software Lifecycle
    • Safety Classes & Level of Concern
    • SOUP and OTS
    • Medical Apps
    • IT-Security
    • Artificial Intelligence
    • And more...
    Artificial Intelligence in Medicine
    Risk Management & ISO 14971
    • Life Cycle Risk Management
    • Update 14971:2012
    • Harm and Severity
    • Hazard and Hazardous Situation
    • Risk Acceptance
    • Risk Mitigation
    • Risk Analysis
    • Software Specifics
    Usability & IEC 62366
    • Main Operating Functions
    • ISO 9241
    • User stories
    • Usability Validation
    Product Development
    • Intended Use Description
    • System Architecture
    • Verification versus Validation
    • Clinical Evaluation
  • Regulatory Affairs
    • Medical Device Regulation
    • IVD Regulation (IVDR)
    • Medical Device Directive
    • Classification
    • CE Marking
    • Conformity Assessment
    • Harmonized Standards
    • MEDDEV 2.7/1
    • FDA
    • And more ...
  • QM System & ISO 13485
    • ISO 13485:2016
    • Quality Policy & Objectives
    • Process Validation
    • Audits
    • Unannounced Audit
    • MDSAP
    • CSV
  • Software & IEC 62304
    • Software Lifecycle
    • Safety Classes & Level of Concern
    • SOUP and OTS
    • Medical Apps
    • IT-Security
    • Artificial Intelligence
    • And more...
  • Artificial Intelligence in Medicine
  • Risk Management & ISO 14971
    • Life Cycle Risk Management
    • Update 14971:2012
    • Harm and Severity
    • Hazard and Hazardous Situation
    • Risk Acceptance
    • Risk Mitigation
    • Risk Analysis
    • Software Specifics
  • Usability & IEC 62366
    • Main Operating Functions
    • ISO 9241
    • User stories
    • Usability Validation
  • Product Development
    • Intended Use Description
    • System Architecture
    • Verification versus Validation
    • Clinical Evaluation

Artificial Intelligence in Medicine

More and more medical devices are using artificial intelligence to diagnose patients more precisely and to treat them more effectively. Although a lot of devices have already been approved (e.g. by the FDA), a lot of regulatory questions remain unanswered.

This article describes what manufacturers whose devices are based on artificial intelligence techniques should pay attention to.

1. Artificial intelligence: what is it?

The terms artificial intelligence (AI), machine learning and deep learning are often used imprecisely or even synonymously.

a) Definitions

The term “artificial intelligence” (AI) itself leads to discussions about, for example, whether machines are actually intelligent.

We will use the definition below:

Definition: Artificial Intelligence

“A machine’s ability to make decisions and perform tasks that simulate human intelligence and behavior. Alternatively

  1. A branch of computer science dealing with the simulation of intelligent behavior in computers.
  2. The capability of a machine to imitate intelligent human behavior”
Source: Merriam-Webster

So it is about machines ability to take on tasks or make decisions in a way that simulates human intelligence and behavior.

A lot of artificial intelligence techniques use machine learning, which is defined as follows:

Definition: Term

“A facet of AI that focuses on algorithms, allowing machines to learn and change without being programmed when exposed to new data.”

Source: Arkerdar: Business Intelligence for Business

And deep learning is, in turn, part of machine learning and is based on neural networks (see Fig. 1).

Definition: Term

“The ability for machines to autonomously mimic human thought patterns through artificial neural networks composed of cascading layers of information.”

Source i.a. HCIT Experts

This gives us the following taxonomy:

Fig. 1: Artificial intelligence is based on numerous techniques, of which machine learning is only one part. Neural networks, deep learning, are part of machine learning.

b) Techniques

The assumption that artificial intelligence in medicine mainly uses neural networks is not correct. A study by Jiang et al. showed that support vector machines are used most frequently (see Fig. 2). Some medical devices use several methods at the same time.

Fig. 2: Most artificial intelligence techniques used in medical devices fall into the “machine learning” category. Neural networks are the second most popular among manufacturers. (Source) (click to enlarge)

 

2. Applications of artificial Intelligence in medicine

a) Overview

Manufacturers use artificial intelligence, especially machine learning, for tasks such as the following:

Function

Data

Detecting a retinopathy

Images of the eye fundus

Counting and recognizing certain cell types

Images of histological sections

Diagnosis of heart infarctions, Alzheimer's, cancer, etc.

Radiology images, e.g. CT, MRI

Detecting depression

Speech, movement patterns

Selection and dosage of medicines

Diagnoses, gene data, etc.

Diagnosis of heart diseases, degenerative brain diseases, etc.

ECG or EEG signals

Detecting epidemics

Internet searches

Disease prognoses

Laboratory values, environmental factors etc.

Time-of-death prognosis for intensive care patients

Vital signs, laboratory values and other data in the patient's records

Table 1: Comparison of the tasks that can be performed with artificial intelligence and the data used for these tasks

Other applications include:

  • Detection, analysis and improvement of signals e.g. weak and noisy signals
  • Extraction of structured data from unstructured text
  • Segmentation of tissues e.g. for irradiation planning

Fig. 3: Segmentation of organs (here a kidney) with the help of artificial intelligence (Source) (click to enlarge)

b) Tasks: classification and regression

The techniques are used for the purpose of classification or regression.

Examples of classification

  • Decision as to whether there is a diagnosis
  • Deciding whether cells are cancer cells or not
  • Selecting a medicine

Examples of regression

  • Determining the dose of a medicine
  • Time-of-death prognosis

3. AI from a regulatory perspective

a) Regulatory requirements

There are currently no laws or harmonized standards that specifically regulate the use of artificial intelligence in medical devices. However, these devices must meet existing regulatory requirements, such as:

  • The manufacturers must demonstrate the benefit and performance of the medical device. For devices that are used for diagnostics purposes, the sensitivity and specificity, for example, must be demonstrated.
  • The devices must be validated against the intended purpose and stakeholder requirements and verified against the specifications (including MDR Annex I 17.2).
  • They must ensure that the software has been developed in a way that ensures repeatability, reliability and performance (including MDR Annex I 17.1).
  • Manufacturers must describe the methods they will use for these verifications.
  • If the clinical evaluation is based on a comparator device, this device must be sufficiently technical equivalent, which explicitly includes the evaluation of the software algorithms (MDR Annex XIV, Part A, paragraph 3).
  • Before development, manufacturers must determine and ensure the competence of the people involved (ISO 13485:2016 7.3.2 f).

b) Regulatory requirements

Manufacturers regularly find it difficult to prove that the requirements placed on the device, e.g. with regard to accuracy, correctness and robustness, have been met.

Dr. Rich Carruana, one of Microsoft's leading minds in artificial intelligence, advised against the use of a neural network he had developed himself to propose an appropriate therapy for pneumonia patients:

“I said no. I said we don’t understand what it does inside. I said I was afraid.”

Dr. Rich Carruana, Microsoft

The questions that auditors should ask manufacturers include, for example:

Key question

Background

How did you reach the assumption that your training data has no bias?

Otherwise the results would be wrong or only correct under certain conditions.

How did you avoid overfitting your model?

Otherwise, the algorithm would only correctly predict the data it was trained with.

What makes you assume that the results are just randomly correct?

For example, it could be that an algorithm correctly decides that an image contains a house. But that the algorithm did not recognize a house, but the sky. Another example is shown in Fig. 3.

What requirements does the data have to meet in order to correctly classify your system or predict the results? Which framework conditions must be observed?

Since the model was trained with a certain quantity of data, it can only make correct predictions for data coming from the same population.

Would you not have achieved a better result with another model or with other hyperparameters?

Manufacturers must minimize risks as far as possible. These also include risks resulting from incorrect predictions made by sub-optimal models.

Why do you assume that you have used enough training data?

Collecting, processing and “labeling” training data is time-consuming. The more data that is used to train a model, the more powerful it can be.

What gold standard did you use when labeling the training data? Why do you consider the chosen standard to be the gold standard?

Particularly if the machine starts to be superior to people, it becomes difficult to determine whether a physician, a group of “normal” physicians, or the world's best experts in a discipline are the reference.

How can you ensure reproducibility if your system continues to learn?

Continuous Learning Systems (CLS), in particular, must ensure that the further training does not, at the very least, reduce performance.

Have you validated systems that you are using to collect, prepare, and analyze data, and to train and validate your models?

An essential part of the work consists of collecting and processing the training data and using it to train the model. The software needed for this is not part of the medical device. However, it is subject to the requirements of the Computerized Systems Validation. 

Table 2: Aspects that should be addressed in the review of medical devices with associated declaration

The questions are typically also discussed as part of the ISO 14971 risk management process and the clinical evaluation according to MEDDEV 2.7.1 Revision 4.

Fig. 4: Input data that only randomly looks like a certain pattern. In this example, a Chihuahua and a muffin (source) (click to enlarge)

c) Approaches to solutions

Auditors should no longer be generally satisfied with the statement that machine learning techniques are black boxes. The current research literature shows how manufacturers can explain and make transparent the functionality and "inner workings" of devices for users, authorities and notified bodies alike.

For example, using Layer Wise Relevance Propagation it is possible to recognize which input data (“feature”) was decisive for the algorithm, e.g. for classification.

Figure five shows, in the left picture, that the algorithm can rule out a number "6" primarily because of the pixels marked dark blue. This makes sense, because with a "6" this area typically does not contain any pixels. On the other hand, the right image shows in red the pixels that reinforce the algorithm's assumption that the digit is a “1”.

The algorithm evaluates the pixels in the rising part of the digit as damaging for classification as "1". This is because it was trained with images where the “1” is written as a simple vertical line, as is the case in the USA. This shows how important it is for the result that the training data is representative of the data that is to be classified later.

Fig. 5: Layer Wise Relevance Propagation determines which input is responsible for which share of the result. The data are visualized here as a heat map (source). (click to enlarge)

Further information

The free online book “Interpretable Machine Learning” by Christoph Molnar, who is one of the keynote speakers at Institute Day 2019, is particularly worth a read.

4. Support

The Johner Institute supports manufacturers of medical devices that use artificial intelligence, for example in the following areas

  • Developing and marketing the device in compliance with the law
  • Planning and performing corresponding verification and validation activities
  • Evaluating the device's benefits, performance and safety
  • Evaluating the suitability of the techniques (in particular the models) and the training data
  • Complying with the regulatory requirements for the post-market phase and
  • Creating tailor-made procedural instructions.

5. Conclusion, outlook

a) From hype to actual practice via disillusionment

Artificial intelligence is currently receiving a lot of hype. A lot of “articles” praise it as either the solution to every medical problem or the start of a dystrophy in which machines will take over. We are facing a period of disillusionment. “Dr. Watson versagt” [“Dr. Watson fails”] was the title on article in issue 32/2018 of Der Spiegel on the use of AI in medicine.

It has to be expected that the media will write over-the-top and scandalized reports on cases where bad AI decisions have tragic consequences. But over time, the use of AI will become just as normal and indispensable as the use of electricity. We can no longer afford and no longer want to pay for medical staff to perform tasks that computers can do better and faster.

b) Regulatory uncertainty

The regulatory framework and best practices lag behind the use of AIs. This leads to risks for patients (medical devices are less safe) and for manufacturers (audits and approval procedures seem to reach arbitrary conclusions).

In 2019, the Johner Institute, together with notified bodies, will publish a guideline for the safe development and use of artificial intelligence - comparable to the IT Security Guideline.

Company

  • About us
  • Contact
  • About Prof. Dr. Johner
  • Our customers
  • Imprint
  • Privacy Policy

Links

  • Consulting
  • Login Auditgarant
  • Trainings
  • Articles

Products

  • Auditgarant (E-Learning)
  • CPMS Seminar
  • Usability Seminar

Free offers

  • Starter-Kit
  • Ask an expert
  • SSRS-Checklist
ISO 13485:2016
Logo Johner Institut

This site uses cookies. By using this website, you agree to the use of cookies.

For more information see our Privacy Policy