MEDDEV 2.7/1 Revision 4: Guidelines for Clinical Evaluations
Revision 4 of the MEDDEV 2.7/1 guideline for clinical evaluations has been in force since 1 July 2016. It offers more detailed assistance than the previous version, but also formulates stricter requirements and surprises with a narrow focus on Europe. Some provisions could turn out to be counterproductive.
The European directive for medical devices 93/42/EEC requires: Manufacturers must provide proof of and document the performance and safety of medical devices in a "clinical evaluation"
The guideline requires a "defined and methodologically sound procedure".
However, the guideline says little about what exactly this procedure should look like. European guideline MEDDEV 2.7/1 provides detailed instructions and thorough assistance. It defines the gold standard and acts as reference for content and structure of clinical evaluations.
This article gives an overview and answers the question: "MEDDEV 2.7/1 Revision 4: fairy godmother or bogeyman?"
New Assistance: MEDDEV 2.7/1
Since 1 July 2016, the guideline's revision 4 has been in force. With its 65 pages, it is much more extensive as the previous version of 46 pages. The enlargement, above all, reflects the writers' fortunately successful endeavor to improve and optimize the assistance and support provided by the guideline. New valuable explanations and tips as well as new helpful appendices can be found.
In terms of regulation, MEDDEV 2.7/1 revision 4 most notably contains aggravations with partly considerable implications. These mainly concern the following topics:
- Qualification of authors: stricter requirements
- Comparability: stricter standards for equivalence of comparable products
- Databases: mandatory search in European databases
- Update cycle: required frequency of updates according to certain criteria
Europeanisation of requirements
The previous version, MEDDEV 2.7/1 revision 3 from 2009, followed the recommendations of then Global Harmonization Task Force (GHTF) to harmonize the requirements and procedures internationally. The new revision 4, on the other hand, contains aspects that imply a Europeanisation of requirements:
- Literature review shall encompass databases with a European focus.
- Reference products shall bear the European CE mark.
In particular the second demand is tough. More about both points below.
The increased rigor corresponds to a general trend. Since recently, a rougher note has been blasted during audits by notified bodies, which are themselves subject to stricter controls.
The background: The public and politicians are calling for greater patient protection, particularly as a result of the breast implant scandal. In this respect, increased market hurdles for medical devices and thus more demanding standards for clinical evaluation seem to be a logical measure to (re-)establish confidence in regulatory processes.
However, the question of whether the new regulations are a success is not determined by the criterion of rigor, but by the criterion of balance: On the one hand, regulations should ensure that only efficient and safe medical devices are on the market. On the other hand, they should not delay and hinder market access more than absolutely necessary. It is not only harmful when unacceptably dangerous medical devices are on the market. It is just as harmful if curative, perhaps life-saving products do not reach the patient because of too restrictive regulations.
MEDDEV 2.7/1 Revision 4: Structure and Content (Overview)
The focus lies on requirements for a report on clinical evaluations as well as on explanations of how to prepare such a report systematically and in a well-planned manner. The working process is understood as an iterative cycle of five stages, the sequence of which should also be reflected in the final report.
- Stage (planning): presentation of the clinical evaluation's objective and structure
- Stage (identification): collection of pertinent data from scientific literature and, if possible, from clinical trials.
- Stage (appraisal): individual appraisal of data sets.
- Stage (analysis): summarizing review of the product in the light of the data.
- Stage (report): evaluation report
These steps are described in detail below.
Appendices of MEDDEV 2.7/1 rev. 4
The annexes contain additional guidelines for specific topics and tasks. These include: A guide to document systematic literature research, strategies for data evaluation, and - finally - a checklist for approval, which is of course also useful for self-checking.
MEDDEV 2.7/1 Revision 4: Conducting a Clinical Evaluation
Objectives and general principles of clinical evaluation (chapter 6)
Clinical evaluation shall prove that the product is efficient and safe. The basis is a set of clinical data, i.e. "the safety and/or performance information that is generated from the clinical use of a device" (MDD 93/42 EWG, article 1.2.k).
Clinical data can originate from the device itself, but also from similar devices for which equivalence must be made plausible. If the available data are not enough, additional clinical data must be generated, e.g. in the context of a clinical trial, especially for new and high-risk products.
Author qualification (chapter 6.4)
Even in MEDDEV 2.7/1 revision 3, the requirements placed on the author of a clinical evaluation were so high that a single person could hardly meet them. The new version requires even more qualifications and knowledge. Amongst others:
- an academic degree plus 5 years relevant work experience (alternatively 10 years relevant work experience without academic degree)
- technology and application of the medical device to be evaluated
- diagnosis and treatment of the disease for which the medical device is intended
- research methodology (the design of clinical trials, biostatistical methods)
- information management and database search
- regulatory requirements
- medical writing
In addition, the authors must make a declaration of interest in which they disclose possible financial or personal conflicts of interest in their relationship with the manufacturer.
The individual author as an all-rounder will become rarer. In the light of the required multi-disciplinarity, flexible cooperation and ad-hoc additions to the team by consulting specialized experts will probably gradually prevail over the individual author.
Report (Chapter 11)
The clinical evaluation's results shall be presented in a report ("clinical evaluation report", CER). According to the latest MEEDEV 2.7/1 revision 4, this report shall, as already mentioned, reflect a systematic sequence of stages which are explained in the following. The report must be included in the technical documentation. The manufacturer must plan and carry out regular updates of the report.
5 Stages of Compiling the Clinical Evaluation
Stage 0 (plan): Definition of the clinical evaluation's focus and context (chapter 7)
The principle: focus, perspective, and context of the clinical evaluation must be presented. This includes, for example: a description of the product to be evaluated and its purpose, as well as the technical and medical background. Based on this, a plan for the clinical evaluation ("clinical evaluation plan") is formulated containing a reference to the fundamental requirements for the fulfillment of which evidence must be provided. It is recommended that you conduct your own literature review for the introductory part on the medical background and treatment alternatives (see below):
Stage 1 (identification): Collecting clinical data (chapter 8)
The principle: Clinical evaluation is based on data which provide information on the device's safety and performance objectively and reliably. Enough data of this kind must first be found or generated. A systematic and traceable procedure shall ensure that data which is advantageous as well as data which is disadvantageous for the device are considered equally.
Source: Suitable clinical data can essentially be obtained from three sources:
- scientific literature
- clinical experiences (amongst others from market surveillance)
- clinical trials
ad A. Search for scientific literature:
Basic idea: The aim is to identify published scientific papers (clinical studies, reviews, meta-analyses) that contain meaningful results on the performance and safety of the product. The literature search should be thorough and, like the subsequent selection process, must be documented comprehensibly.
Procedure: Key component of collecting literature is the systematic search for published clinical studies, reviews and meta-analyses of the product or of equivalent products in scientific databases.
Latest requirement: Europe-centric literature search (appendix 4)
MEDDEV 2.7/1 revision 4 contains an important amendment: Up to now, PubMed has often been chosen as the single source for the main search as the probability of finding top-ranking studies in said database is high. This was often regarded as sufficient and was accepted. MEDDEV 2.7/1 revision 4 now demands to especially search for European data which is not always made available in databases such as PubMed. The reasoning for this: "Studies yielding negative results or user experience (...) may not qualify for publication in high impact medical journals. Low impact journals available to European users and other sources may therefore need to be searched." Explicitly recommended as a European database is Embase.
New requirement: equivalent devices must be CE marked (appendix 1)
Not only clinical data on the product itself, but also data on sufficiently similar reference products may be used to evaluate performance and safety. The degree of equivalence must be shown in respect to three aspects: clinical, technical and biological aspects. In the latest MEDDEV 2.7/1 revision 4, the definitions and criteria for this are pointed out more clearly and are formulated in more detail and more precisely but are in some cases somewhat stricter than in the previous version.
However, having read the explanations and requirements for clinical, technological, and biological equivalence, one stumbles across another Euro-centric amendment: According to new MEDDEV 2.7/1 revision 4, data on equivalent devices are only suitable if they were collected using CE-labeled devices. That is to say: Only devices meeting the European market's fundamental requirements shall be used as reference device. The reason for this: not only the device, but also the patient treated with it shall be considered when conducting a clinical comparison. A device for the European market is for European patients.
Exceptions shall only be possible if it can be proven that clinical data on the reference product can be transferred to Europe. In the pharmaceutical world, this is a long-established requirement. However, it is questioned for whether it is actually appropriate for all medical devices.
Another aggravation will make it even harder to refer to equivalent products in the future: Equivalence must be explained much more comprehensively and in more detail than it was previously required, partly based on information that can hardly be obtained other than from the manufacturer of the competing product itself (details of manufacture, opposing measurements of various kinds, comparative product drawings, etc.). Even if a reduced comparability assessment of notified bodies may still be accepted within the framework of revision 4, it is clear that regarding MDR, this possibility will not apply to implants and Class III products, at least.
Sequence of literature search and selection (chapter 8.2)
For the literature research, a methodical approach is essential: Starting from an appropriate research question, it is important to select suitable databases and to finally formulate promising search terms. Apart from database search, informative publications can also be found using bibliographies, targeted manual searches or in the manufacturer's catalog. In the final report, search strategies and particularized results must be documented in detail - just as the process of selecting "relevant" literature outlined in the following figure.
ad B. Clinical experience:
Further clinical data result from the use of the product or of equivalent products which are acquired for instance from the manufacturer's market surveillance report, registers, unpublished observational studies on individual patients or cohorts by the manufacturer, as well as from databases containing reported "adverse events", product warnings etc.
ad C.Clinical trials:
Clinical trials are necessary if there is insufficient meaningful data available for a medical device.
Stage 2 (appraisal): Individual evaluation of clinical data (chapter 9)
Main idea: On the one hand, it is important to sift out those data from the wealth of data that can contribute to the topic and the clinical evaluation. On the other hand, quality and significance of the data left in the basket after the selection process must be assessed substantially and must be documented.
New to revision 4 is the explicit distinction between "pivotal data" and "other data": "When evaluating the relevance of collected data it is important to consider whether the data are intended to directly demonstrate adequate clinical performance and clinical safety of the device (often referred to as pivotal data), or whether the data serves an indirect supportive role." Hence, the search for informative data comprises two different search requests, namely (1) the search for information on the medical and technological background, state of knowledge, and state of the art as well as (2) the search for studies contributing directly to the assessment of the device's clinical performance and safety. This has largely been done so far, but was not expressed by MEDDEV, or at least only very vaguely.
Procedure: There is no universal method for filtering, selecting and evaluating. Professional competence and judgement are indispensable. However, the reader of the final report must be able to understand the selection and evaluation of the data in detail.
Obviously, literature research yields many hits which have nothing to do with the topic. By means of title and maybe abstract, these are immediately sorted out, just as studies of obviously insufficient quality. What remains are the "possibly relevant" hits. These are now examined in full in a second round. Articles which turn out to be unrelated or of little meaning after this more detailed review are again excluded, whereby the exclusion must be justified in each case. The result of the selection is the final collection of "relevant" literature, the list of studies discussed and referenced in the final report.
Individual evaluation: "Relevant" studies are further evaluated in detail to be able to assess as accurately as possible what the results reveal about the device's performance and safety. Thereby, an important criterion is the level of evidence, which, according to current conventions, essentially reflects the study design. Here, multi-center double-blind studies and good meta-analyses receive the highest rating. Case reports score correspondingly lower; expert opinions are in the lower zone. Equally important is the degree of equivalence already discussed, which indicates how similar the product examined in the study is to the product to be evaluated. Patient numbers, endpoints, detailed results and conclusions are also included in the evaluation, as are observed side effects. The studies' design and statistical analyses are to be evaluated against strict criteria.
Stage 3 (analysis): Summarizing evaluation of the Device (chapter 10)
The summarizing analysis of all relevant data answers the following questions:
- Does the clinical data collectively prove that the medical device is effective and safe when it is used according to its intended purpose?
- Are remaining risks acceptable in view of the benefits?
This also includes the evaluation of side effects (Essential Requirement no. 6): undesirable side-effects may not constitute unjustifiable risks when weighed against the intended performance.
Analysis and inferences should be detailed and comprehensible, including an extensive and critical consideration of all data reviewed.
MEDDEV 2.7/1 revision 4 also offers detailed new instructions, recommendations and tips for this work package, which in terms of content, however, essentially correspond to current regulations and practice. Amongst others, the requirement to demonstrate compliance with Essential Requirements, particularly regarding clinical performance, safety and the positive risk-benefit ratio was elaborated more precisely.
Stage 4 (report): Report on the clinical evaluation (chapter 11)
The report documents in detail and comprehensibly the course, argumentation and conclusions of the clinical evaluation. In particular, the explicit recommendation to build the report based on consecutive work packages ("stages") was added in MEDDEV 2.7/1 revision 4. Appendix 9 contains a respective structural template for the report which also considers the above reference to the Essential Requirements. Moreover, appendix 10 offers a checklist for the clinical evaluation's release which can be used to systematically review both regulatory as well as substantive issues at the end of the Clinical Evaluation report.
Regular updates (chapter 6.2.3)
A clinical evaluation is a "living document" and should be regularly updated. Whereas MEDDEV 2.7/1 revision 3 still left quite open what is meant by "regular", the fourth revision now makes things clearer.
The clinical evaluation is updated:
- if the manufacturer receives new information from the market (PMS) which could change the current evaluation;
- if there are no such information, at least once a year if the product is associated with significant risks or if it is not sufficiently established;
- or every 2 to 5 years if the product is not associated with significant risks and sufficiently established. A justification for this should be provided.
MEDDEV 2.7/1 Revision 4 Appendices
Requirements and recommendations regarding specific topics can be found in the appendices.
- Appendix 1: Demonstration of a reference product's equivalence
- Appendix 2: When are additional clinical investigations necessary?
- Appendix 3: Typical contents of a device description
- Appendix 4: Sources of literature
- Appendix 5: Literature search and review
- Appendix 6: Appraisal of clinical data - examples of studies that lack scientific validity
- Appendix 7: Analysis of clinical data - compliance to specific Essential Requirements
- Appendix 8: Devices for unmet medical needs - aspects to consider
- Appendix 9: Clinical evaluation report - proposed table of contents with examples of contents
- Appendix 10: Checklist for the release of the clinical evaluation report
- Appendix 11: Information on declarations of interests
- Appendix 12: Activities of notified bodies
What Are the Consequences?
For the author or the team of authors, it will altogether become more time-consuming to compile a clinical evaluation. The increased requirements for the presentation of the equivalence of comparison products and the individual appraisal of clinical data play a significant role in this.
For manufacturers, the consequences may be more serious, especially because the argumentation based on comparable products is often difficult or even impossible. "Product comparison is dead" - we can already hear it from the notified bodies. Indeed, if the new requirements are strictly implemented, many or even all the comparable products that have previously been approved may not be used anymore. The consequences are serious: literature will be much less likely to provide enough data to prove the performance and safety of a product. This means that clinical trials are to be conducted much more frequently, which is associated with more effort and costs for the manufacturers concerned.
Our opinion: Patient protection has the highest priority. The massive restriction of the permitted comparable products could seem excessive and make little sense if the consequences are considered. Often, this regulatory tightening will mean that valuable information must be waived for formal reasons. On the one hand, it therefore contradicts the requirement to use "all relevant clinical data" for evaluation purposes. On the other hand, the additional costs for clinical trials, especially for smaller manufacturers, will sometimes be difficult or impossible to bear. Against this background, the regulation may be counterproductive: medical devices that could be beneficial or even life-saving are delayed or not put on the market at all and are thus withheld from patients. Here, it seems as if the sense of proportion for the balance between patient protection and access to remedies has been lost. It cannot be emphasized enough: People can be harmed not only by defective medical devices, but also by the lack of healing or even life-saving medical devices.
Updates on MEDDEV 2.7/1 Revision 4
We are observing that notified bodies are interpreting the new MEDDEV very strictly. Notified bodies demand strict conformity with this requirement even in the case of uncritical class I devices and in the case of products which have been on the market for decades.
At a meeting of the notified bodies in April 2017, the following agreement was reached:
During the first week of April 2017, the European Notified Bodies, representatives from Industry associations and member states met in Brussels in order to agree on the application of the Clinical Evaluation Guidance Document with the following outcome:
The main focus concerning clinical evaluations is that the manufacturer must use a scientifically sound method to continuously show and confirm safety and performance as required by the European directives. MEDDEV 2.7/1 revision 4 was published last year without official transition period. To note, guidance documents do not have a legally binding character and other methods may be applied. However, if a manufacturer claims compliance to MEDDEV 2.7/1 revision 4 to fulfil the state-of-the-art requirements of the directives, compliance to the selected guideline is expected.
Regarding a transition time, no general agreement was achieved. Now the focus is on practical approaches in collaboration of the different interested parties (Member States, Notified Bodies and Manufacturers) – always taking into account the above-mentioned compliance with the European directives (or soon European regulation). As far as the intended update of MEDDEV 2.7/1 is concerned, it can be expected that there will be none planned in the near future.
The rigid interpretation of the requirements for clinical evaluation sometimes presents manufacturers with an unsolvable task. For consulting companies, this increases the effort required to prepare clinical evaluations for our customers. Click here to find out more about how we can support you with conducting clinical evaluations.
Fairy godmother and bogeyman - the new MEDDEV 2.7/1 revision 4 is a bit of both.
First of all, it is pleasing to see the large number of well thought-out and detailed aids and tips for carrying out a clinical evaluation. The stricter requirements improve safety and thus patient protection.
However, some of the requirements and in particular the massive restrictions on the permitted reference products appear counterproductive, as they ultimately make access to helpful products unnecessarily difficult or even impossible with the generally commendable goal of patient protection.
Before the forthcoming European Medical Device Regulation, the implementation of the MEDDEV guideline's fourth revision may transitionally be demanded by notified bodies in weakened form. Thereafter, most requirements will be anchored in the new regulation and leave no margin, so that manufacturers would be well advised to implement revision 4 as a first step towards compliance with the new legislative framework.
The article was written in collaboration with Dr. Franz Mechsner, consultant for Clinical Evaluation at the Johner Institute, and Florian Tolkmitt, expert for Regulatory Affairs & Clinical Evaluation
Please contact us if you have questions or need support to compile a compliant clinical evaluation report.