Package leaflets (or “patient information leaflets”, as they are called in the UK) serve patients as an important source of information to learn about the drugs they are using – and unlike a doctor, nurse, or pharmacist, they are always at hand when the patient has a question. If a patient does not understand the leaflet, the safe use of the drug may be jeopardized. Moreover, studies have shown that around 30 % of the patient population are left feeling insecure and almost the same percentage have discontinued or even not started to take a drug at least once[1] because of the way leaflet information was presented. Therefore “patient-friendly” leaflets not only increase the safety of use, they can actually improve the uptake of prescription drugs.
Since the early 1990s the guidelines issued by the European Commission have essentially determined the contents of package leaflets. In the 1998 Guideline on the Readability of the Label and Packaging Leaflet of Medicinal Products for Human Use, the Commission added to its agenda a readability criterion, that is a statistically verifiable procedure that would test the readability of package leaflets.[2] This was followed by EU Directives 2001/83/EC and 2004/27/EC, which made readability tests a fundamental prerequisite for drug approvals, a legal requirement in the EU since late 2005. Only applications referring to very similar approvals such as line extensions that use the same mode of administration are exempted from this obligation.
Several procedures are available for readability tests and they include patient interviews, written tests, and content analyses using the methods of communication science. The patient interview (often referred to as the “Australian model”[3] is the only procedure to be explicitly described in any guideline and as a result has become the de facto standard associated with a high degree of acceptance by authorities, although other readability test procedures may be used as well.[4] As with any testing procedure, the readability test should be in keeping with recognised standards and be proven to yield valid and reliable results.
Despite their apparent dependence on the individual test subjects, Readability User Tests can (and should) be validated: by conducting a set of meta-tests of the validity, reliability and responsiveness of the user test in question.
A test can be called valid if it detects, in a reproducible way, difficulties in finding or understanding information during the interview, without the result being significantly influenced by external factors such as the subjects’ education or profession. A feasible meta-test thus consists in presenting an identical questionnaire to two groups of participants that differ in medical knowledge and/or reading background. The questionnaire and the interview procedure must be able to detect areas of difficult information within the package leaflet consistently, and the influence of the external factors between both groups must be shown to be low. If this is the case, the questionnaire can be considered valid for the problem (Table 1).
If intra-individual variations can be shown to be low for repeated applications of the test, it is considered reliable. This can be shown by test subjects having to answer the same questions for the same package leaflet after a time interval of approximately two to three months. This time span can be considered long enough to prevent test subjects from recalling their previous answers from memory. If the questionnaire and the interview procedure does indeed show low intra-participant variability, the questionnaire can be considered reliable.
Finally, the procedure has to be responsive enough to measure whether modifications such as the substitution of conversational words (eg, “accumulation of fluid”) for specialised terms (eg, “oedema”) result in significant changes in readability or comprehension. For this meta- test, a regular user test is carried out. It measures how well the test subjects are able to locate information and how well the found information is understood. After a sufficiently long time span, the same test subjects receive a second package leaflet. This package leaflet is almost identical to the first one, except that about half of the asked- for information is changed in wording or placement, thus introducing artificial difficulties. For example, medical terms are used instead of explanations. If the questionnaire is able to detect the information that has been changed between the test runs, while yielding similar results for questions regarding unchanged information, it is shown to be responsive (Table 2).
Maintaining highest quality standards is a matter of course where the safe use of medicinal products is at stake. The validation procedures for readability user tests outlined above provide but the first steps towards attaining this goal.
Subject recruitment
Both the “readability guideline” of the European Union and individual EU Member State authorities have given concrete requirements for the interview procedure. At least 20 subjects should belong to the drug-target group. This sets a challenge to find suitable test subjects, especially as the group must be balanced in other ways such as age distribution and levels of education. To prevent bias from participant learning effects (or similar), subjects should participate in readability tests only once or with a long interval between tests. Subjects should also be excluded if they have a fundamentally negative attitude towards package leaflets. Products used only by health professionals might also be evaluated, although interestingly in this case, the test subjects remain patients rather than the health professionals.
Questions
A Readability User Test takes the form of an interview that comprises 12 to 15 questions[5] covering all the important and safety-relevant issues of the respective package leaflet. The selection of questions should be standardised (Table 3) in such a way as to ensure that the whole leaflet is placed under scrutiny. Visual clarity and layout of the text and the package leaflet is also tested.
Conducting the interview
The EU Guideline example test states that each interview should take between 30 to 45 minutes (any longer and the subject is likely to grow weary).The test subjects are asked to look at the leaflet to find answers to questions such as “Is this medicine safe to be taken by a pregnant woman?” (Chapter of package leaflet: warnings) or “What is the normal daily dosage” (Chapter of package leaflet: Dosage).The subject has the package leaflet to refer to throughout the interview but when they find the information they are encouraged to repeat it to the interviewer in their own words – this can indicate to the interviewer that the subject understands the information and is not merely “parroting” it. The subjects’ answers and statements are recorded, and the ease of finding, understanding and using the information is evaluated for each question.
The “plain text” of a package leaflet (ie, without its final layout or graphics) is not adequate for performing readability tests. Tests are required by the authorities to be performed with either the original package leaflet (if the product is already on the market and is being retrospectively tested) or if that is unavailable with a “mock-up”. The mock-up should be equivalent to the a real leaflet in respect of contents, design, type of paper, folding pattern and font size (currently at least 8 Didot points, perhaps 12 Didot points in the future[6]. In this regard there is a clear dependency from font type (Table 4).
Before the test, it is recommended that a pilot be performed with three to six subjects. The pilot study will detect serious defects in the package leaflet on the one hand, but it will also check the questionnaire for its aptness. If early participants find the leaflet too challenging then it is necessary to inform the licensing applicant and suggest suitable changes. The pilot phase thus makes it possible to optimize the test and to avoid unnecessary investments of time and money in the main test rounds.
Following on from the pilot, the actual readability test is performed over at least two test rounds and evaluated after each round with ten participants. However, a responsible CRO will evaluate the results after each interview – and recommend a temporary halt to the test if improvements to the leaflet can be made. Since the CROs are exposed at first hand to the subjects’ reactions to the leaflet it is recommended that they are involved in performing any necessary editorial revisions. This necessitates a close collaboration between the service-company and the licensing applicant, and the CRO may assist in responding to requests for further information from Competent Authorities, if requested by the applicant.
A package leaflet is considered usable if 90 per cent of the requested information is found, and 90 per cent of the information found is understood. In its current version, the EU Guideline offers some flexibility: “The objective is to have at least 16 out of 20 consumers able to answer each question correctly. However, it is not necessary for the same 16 people to answer each question correctly.”[7]
But, the EU Guideline is currently being updated; a departmental draft says about the comprehension rate: “A satisfactory test outcome for the method outlined above is when 90% of literate adults are able to find the information requested within the Package Leaflet, of whom 90% can show that they understand it.”[8] This means a significant tightening of the criteria for reaching the targets of readability if it is implemented, or in other words in the future, the same 16 of 20 subjects would have to answer all questions correctly (Table 5).
If such a change becomes permanent, it would suggest that the European Commission is cloning “word for word” the official guidelines of the UK authority, the MHRA (Medicines and Healthcare Products Regulatory Agency). The MHRA’s requirement that the same 16 of 20 subjects have to answer all of the questions correctly is a stricter criterion that is suitable for the shorter patient leaflets once common in UK countries. With longer patient leaflets and their extra information, the MHRA’s stringency lacks diagnostic power and readability tests turn into an examination of the intellectual capability of their test subjects rather than the clarity of the leaflets. Suffice to say, the European Commission’s final decision on the future criteria for readability testing is awaited with eager anticipation by many people across the industry.
The Guideline specifies that in all European marketing authorisation procedures (Centralised, Decentralised and MR) the final report of a readability test has to be submitted in English. In national marketing authorisation procedures, the respective native language is to be used.[9] In terms of readability testing the Guideline specifies that any official language of any of the countries of the European Union can be used[10] but it has to be guaranteed that the contents of the tested version of the text are accurately translated into all the languages that are spoken in the areas where the drug is to be marketed. In practice, patient leaflets for European marketing authorisation procedures are mostly tested in English to facilitate the procedure, even if the cost may be slightly higher.
There are “four golden rules” for package leaflets. Firstly, the language should be as clear and simple as possible; technical terms and foreign words can be given as supplementary explanations but placed in parentheses. Secondly, given the size of most leaflets, cross- references tend to make it more difficult for the patient to find the information (however, every effort should be made to avoid doubling of information). Thirdly, contents of the package leaflet should be limited to the information that is really relevant for the patients (the laboratory values that the physician will have to monitor should be in the summary of product characteristics - not in the leaflet!). Finally, important information should be highlighted in bold print – a very obvious point indeed, but it is astounding how often test subjects remark upon the lack of bold print in the leaflets they are shown.
A valid and reliable readability test not only makes patient information more comprehensible to patients, it also helps to make the actual taking of drugs safer and more efficient. The current EU Guideline’s readability evaluation criterion[11] does a good job measuring the comprehensibility of package leaflets (a fact that the MHRA may be accepting as it seems to be relaxing its stricter criteria). Hopefully, the latest revision of the EU Guideline will not be regressive in terms of adopting a stringency that was always better served on shorter Anglophone leaflets. The aim of the authorities after all should be two-fold: encouraging clear and easy-to-read leaflets on the one hand, and also setting realistic goals and procedures to follow across the EU.
[1] Nink, K, Schröder, H. (2005): Zu Risiken und Nebenwirkungen: Lesen Sie die Packungsbeilage? Wissenschaftliches Institut der AOK.
[2] Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, September 29, 1998.
[3] Sless, D., Wiseman, R. (1997): Writing about medicines for people: Usability guidelines for consumer medicine information, Canberra.
[4] Draft Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, Chapter 3, Section 3.2, Revision September 2006.
[5] European Commission (2006): Guidance concerning consultations with target patient groups for the package leaflet.
[6] cf. Draft Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, Revision September 2006, Chapter 1.
[7] Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, September 29, 1998, Annex 2.
[8] Draft Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, Revision September 2006, Annex 1.
[9] Draft Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, Revision September 2006, Chapter 3.
[10] cf. ibid.
[11] Guideline on the Readability of the Label and Package Leaflet of Medicinal Products for Human Use, 29 September 1998.