Afamelanotide for treating erythropoietic protoporphyria

Burden of disease

4.1 The patient experts explained that phototoxic reactions can be triggered by even a few minutes of exposure to light, particularly when light is at its most intense on sunny days in the summer, and that the reaction itself lasts from 2 to more than 10 days. They described the pain during a reaction as intense, intolerable and not relieved by pain medication. Furthermore, the pain is neuropathic, meaning that even a light touch to the skin during a reaction exacerbates the pain. The patient experts also reported an all-encompassing tiredness associated with a phototoxic reaction, which can take weeks to resolve. Sometimes, the phototoxic reactions are accompanied by redness and swelling resembling a second-degree burn, but often there are no external signs. The committee recognised that phototoxic reactions cause serious and severe symptoms, including intense pain and extreme tiredness, that last for days.

Effects on day-to-day activities

4.2 People with erythropoietic protoporphyria (EPP) describe the symptoms of phototoxic reactions as being debilitating, preventing them from being able to do day-to-day activities. They also highlight that, without anything to treat the pain or the phototoxicity, their only option is to wait for the phototoxic reaction to stop and their bodies to heal. The patient experts explained that, because phototoxic reactions are unbearable, they will do anything it takes to prevent them. In the absence of any treatment that prevents phototoxicity, this involves avoiding light. They described how they need to constantly assess the light conditions and take measures to minimise the risk of a phototoxic reaction. They explained that various sources of light can cause phototoxic reactions, including some artificial lights and daylight through glass, which makes light avoidance very difficult. They also described how short light exposure triggers nerve stimulation (a prodromal phase) that prompts them to quickly withdraw from further exposure. They explained that assessing, avoiding and withdrawing from light becomes automatic and ingrained (or 'conditioned') behaviour. This, and the fear of a phototoxic reaction, are major and constant causes of anxiety. The committee understood the significant impact of EPP on day-to-day activities.

Psychological and stigmatising effects

4.3 The patient experts commented on further psychological effects of the condition, including post-traumatic stress, flashbacks and nightmares. They explained that suicidal ideation is not uncommon and can emerge from a young age. People with EPP also face difficult situations because other people do not understand the condition. They are often bullied or harassed. They may face hostility and disbelief about their symptoms. They often feel isolated. Also, other people may fail to make appropriate allowances for the condition. In response, people with EPP try to conceal their difficulties and their condition. The committee acknowledged that the psychological and stigmatising effects of the condition are striking and significant.

Far-reaching effects on the lives of patients and their families

4.4 People with EPP report that they often turn down invitations to activities or events. This leads to feelings of social isolation and compromises family life because they cannot take part in outdoor activities or go on holidays. A patient expert explained that his children cannot understand why he cannot join in, which leads to guilt and depression. The patient experts stated that they have had to adapt their careers to manage the measures they need to take to avoid light. The British Porphyria Association stated that its members reported choosing jobs that are indoors with minimal travel and even night jobs to minimise light exposure. A study from Holme et al. (2006) reported that most people with EPP were in employment or education, but that 47% (n=60/127) of those in work felt their choice of occupation had been influenced by their condition. Education choices are similarly affected. The British Porphyria Association stated that, for some families, the children may take on caring for a parent with EPP or other responsibilities that the parent cannot do because of their EPP. It also noted that EPP can place a financial burden on families because of loss of earnings and the expense of measures to protect against sun exposure. One clinical expert explained that EPP either causes debilitating pain if people with the condition try to live a normal life, or anxiety and isolation if they try to avoid the pain by staying indoors. Testimonies received during consultation emphasised the extent of the burden of the condition, including the physical pain from light exposure, and the severe anxiety, stigmatisation, psychological effects and social isolation from having to avoid light. The committee recognised that EPP is a serious, debilitating and disabling condition with far-reaching effects on the lives of patients and their families.

Current treatments

4.5 There is no effective treatment for the underlying cause of EPP, to protect against phototoxicity or to relieve the pain it causes. The only way patients can avoid phototoxicity is to avoid light (see section 4.2). The clinical experts stated that beta carotene and narrow band UVB therapy have been tried as treatments to prevent phototoxicity. However, they explained that these are used less and less because of a lack of clinical effectiveness and because of associated adverse effects (such as an increased risk of death from lung cancer and cardiovascular disease with beta carotene, and an increased risk of developing skin cancer with narrow band UVB). Light avoidance and covering the skin are the only options available to people with EPP. One clinical expert noted that light blocking creams like Dundee cream do not completely block light, and are also not ideal because they are noticeable on the skin. The committee concluded that there is no effective treatment for preventing phototoxicity caused by EPP, so there is an unmet need for an effective treatment.

Diagnosis

4.6 The committee highlighted that, like with many rare conditions, people with EPP have experienced delays in getting a diagnosis. The British Porphyria Association stated that the median age of diagnosis is 22 years, even though the age of onset of EPP is at birth or soon after for most people. One reason for this is that awareness and knowledge of the condition is very low, both among the public and in general medical practice (outside of specialist porphyria centres). People with EPP have reported that the lack of understanding about the condition and their experiences (see section 4.3), and delayed diagnosis, have meant they have the condition without support for many years. The committee recognised that delay in the diagnosis of EPP is a problem.

Variation in symptoms

4.7 The committee explored the variation in symptom severity in people with EPP. One clinical expert stated that most people (around 70) under his care have 'classical' EPP, and that they could have between 2 minutes and 40 minutes of sun exposure before having a phototoxic reaction. However, the pain severity and duration of a phototoxic reaction have less variation between people than the sun exposure times (that is, phototoxic reactions are similar in severity and duration across people with classical EPP). This expert also noted that he had treated mild EPP in around 16 people who could be in very strong sunshine for several hours without a phototoxic reaction. Both clinical experts stated that people with mild EPP may not need, or choose, to have afamelanotide. The company stated that it is not possible to measure the severity of EPP. The committee acknowledged that there is some variation in how long people with EPP can be exposed to sunlight without a reaction. It concluded that any variation in patient experience of the condition was unclear because of a lack of data.

Features of EPP that make decision making difficult

4.8 The committee recognised that EPP is a unique condition with unique challenges and effects for patients. It acknowledged the nature of EPP as a disability, and understood its duties under the Equality Act. It considered in detail which features of the disability associated with EPP might cause people to be disadvantaged within a highly specialised technologies evaluation. It also considered how it would be reasonable to adjust its approach to avoid discrimination and promote equality. The committee considered that 1 of the main features that might affect its approach to decision making, and the need for reasonable adjustments, was the specific challenge in measuring the effect of the condition and its treatment on quality of life (QoL; see sections 4.13 to 4.16). It heard that there was an important lack of robust scientific instruments to measure such effects. It also heard in detail how the ingrained, automatic behaviours to avoid exposure to light ('conditioned light avoidance behaviours'; see section 4.2) further contribute to the substantial challenges in measuring and quantifying the effect of treatments for EPP in clinical studies (that is, even with treatment people may continue to avoid light for some time because of these conditioned behaviours). It recognised that challenges of measurement of treatment effect are seen in other conditions encountered in highly specialised technology evaluations, and are in that sense not unique to EPP, so it should not deviate entirely from its normal approach. However, given the particular issues associated with EPP, it would nevertheless be appropriate to take these challenges into account. The committee concluded that it would take into account the nature of EPP as a disability throughout its decision making, and how it would be appropriate to adjust its approach in the context of this disability.

Role of the committee

4.9 The committee explored in detail the clinical-effectiveness evidence for afamelanotide. The company highlighted that it had been through a long and complex regulatory process and that, based on input from patient and clinical experts, afamelanotide had been granted a marketing authorisation under exceptional circumstances. This was because the European Medicines Agency (EMA) recognised that the comprehensive data on the efficacy and safety required for a regular marketing authorisation could not be generated, but that the benefit–risk balance based on the evidence available was favourable. The company stated that the evaluation committee should not reopen the conclusions made by the EMA's Committee for Medicinal Products for Human Use about the efficacy of afamelanotide. The committee emphasised that it did not seek to re-examine the conclusions of the EMA. The committee instead considers the potential benefits (effectiveness), costs and uncertainties around recommending mandatory funding of a technology (in this case, afamelanotide). The committee concluded that it was appropriate to consider the clinical effectiveness of afamelanotide and the uncertainties in the evidence base as part of its decision making.

Clinical-effectiveness evidence

4.10 The committee noted that there were 4 randomised placebo-controlled trials of afamelanotide:

4.11 The committee was concerned to note that the company's submission did not include complete trial details, such as full baseline data. It meant that the ERG was unable to independently assess the methods and reliability of the clinical-effectiveness assessment of afamelanotide in the trials. The committee understood that the ERG had, when possible, extracted data from publications available to supplement the information available in the company's submission. The ERG pointed out that the Good Clinical Practice (GCP) inspection done by the EMA and reported in the public assessment report highlighted concerns with CUV029 and CUV030, including unsatisfactory collection and analyses of data. The committee acknowledged that there were potential limitations in some of the clinical trials.

4.12 The committee also considered evidence from observational studies, patients' and experts' testimonies, and additional evidence described by the clinical experts.

Challenges of evidence collection

4.13 The company explained that quantifying the effects of afamelanotide was highly challenging. This was because of a lack of scientific tools to capture the true impact of EPP and so the benefit of afamelanotide, rather than problems with the trials. The committee understood that QoL was measured in the clinical trials using both generic tools (short-form 36 [SF‑36] and Dermatology Life Quality Index [DLQI]) and a novel condition-specific measure (EPP‑QoL), but that all of these had limitations (see sections 4.17 to 4.19).

4.14 The committee was aware that measuring the effects of afamelanotide through light exposure times was affected by averaging – that is, the light exposure times reported in the clinical trials were averaged both between people and over time. Therefore, exposure times were influenced by factors such as people's occupations, their daily activities, the weather and conditioned light behaviour. It agreed that this could have affected the trial results, but was unclear to what extent.

4.15 Clinical and patient experts discussed how long it would take for patients to begin to unlearn conditioned light avoidance behaviour. They explained that, for most patients, it usually takes up to 2 to 3 implants to start to unlearn behaviour and increase the amount of time spent in light. For a minority of patients, it may take only 5 to 6 weeks to start increasing light exposure. A clinical expert stated that some of the trials (including CUV017 and CUV029) may have been too short for patients to have changed the conditioned light avoidance behaviour. Moreover, the experts highlighted that some changes (including overcoming psychological barriers and changing employment) may take years to arise.

4.16 The committee acknowledged that there are challenges associated with measuring the effect of EPP and the benefits of afamelanotide, and that these contributed to uncertainties in the clinical trials.

Measuring quality of life

4.17 Recognising the challenges in capturing the impact of EPP on QoL, the committee discussed in detail the QoL measures that were included in the clinical trials. Patient and clinical experts highlighted that factors such as fatigue, particularly impact on the lives of patients and their families, and that the effects of stigma may not be fully reflected in any of the QoL measures.

4.18 The condition-specific QoL questionnaire, EPP‑QoL, was developed by the company. The committee acknowledged that, to be appropriately validated, a questionnaire should be able to support labelling claims granted by the EMA and the US Food and Drug Administration. The company stated that it had consulted with EPP experts to develop the EPP‑QoL. However, it was unable to provide the committee with a response to whether it had used standard methods for developing and validating this tool. Furthermore, the EPP‑QoL was modified while the trials were ongoing and data was being collected, and some questions were removed (although the clinical experts explained that there was evidence that removing some questions had not affected the results). The committee was particularly concerned that a question relating to capacity to go to work or school was removed from the EPP‑QoL. It also worried that there were no questions relating to the effect of pain. People with EPP stated that these aspects are of great importance to them. The company stated that it had not included a question on how pain affected patient's QoL because pain is not considered to be comprehensive in describing symptoms during a reaction. The company also stated that, because patients avoid light, it is rare for them to experience pain, so measuring it would not yield useful results. The committee appreciated the nuances of capturing the burden of the condition because of light avoidance but, based on extensive patient testimonies, it maintained that pain was an important outcome. One clinical expert added that, because of small numbers of patients, there was a limit to how much the EPP‑QoL could be optimised, and also that seasonal variations were important in interpreting the results. They explained that, ideally, a QoL assessment should be done during each of the 4 seasons to capture these variations. The committee considered that any QoL measure should capture the aspects of the condition that affect a person's QoL and, for EPP, this should capture QoL during and between phototoxic reactions. It also considered that the EPP‑QoL did not appear to capture some aspects of EPP that people with the condition and their clinicians report as important. However, there was substantial feedback from stakeholders that the EPP‑QoL is a relevant tool. Also, the clinical experts considered that the EPP‑QoL was the best tool to date to capture the burden of EPP. The committee concluded that the EPP-QoL provided relevant evidence that it would take into account in its consideration of the clinical effectiveness of afamelanotide. After the second consultation, the committee was aware of a new study (Biolcati et al. 2021) which reported a validation of the EPP‑QoL. This study provided evidence for the reliability and validity of the 10‑item symptom score. But it did not find support for 2 questions on wellbeing, which the committee considered limited the tool. The committee concluded that there was still some uncertainty about how the EPP‑QoL could be interpreted and whether it would reliably capture all treatment benefits with afamelanotide. But it noted that it had been partially validated. The committee was also aware that the EPP-QoL had not been assigned preference weights and had not been mapped to an outcome measure that could provide preference weights. This meant that the measure could not be used to generate utility values (see section 4.36).

4.19 The committee noted that the SF‑36 and DLQI had been used in some of the clinical trials. However, the company stated that it had received advice from clinical experts that these measures were not appropriate for capturing the QoL of people with EPP. The DLQI is a validated QoL questionnaire, but is validated for conditions only affecting the skin and not for EPP. The ERG considered that, although not perfect, the DLQI addresses some factors that affect the QoL of a person with EPP, such as pain and ability to work or study. The patient experts explained that the DLQI includes questions that are not relevant to EPP, such as feelings of embarrassment or self-consciousness relating to skin conditions, and that it does not capture non-skin components of EPP such as fatigue. The clinical experts explained that the DLQI does not ask anything about exposure to light, unlike the EPP‑QoL. Furthermore, the company stated that the DLQI does not ask about feelings of anxiety. The committee noted that, in a large observational study, DLQI had been shown to be sensitive to the effect of EPP on people with the condition. However, the same issue that was seen with EPP‑QoL on seasonal variations (see section 4.18) applied to interpreting DLQI scores. The committee concluded that, although DLQI had notable limitations, it had been one of the tools incorporated in the clinical trials at the outset to measure QoL and the results were relevant to its consideration of clinical effectiveness. The committee also stated that it was disappointed that available SF‑36 data had not been presented by the company because this measure includes questions on fatigue and anxiety that are not captured by the DLQI.

4.20 After the second consultation, the committee considered QoL evidence submitted by the International Porphyria Patient Network (IPPN). This consisted of EQ-‑D data collected from a feasibility study carried out in Switzerland, and from an informal patient survey involving people with EPP from 4 European countries. Neither of these studies had been peer-reviewed.

Feasibility study

4.21 The feasibility study included 5 patients with EPP who had been having afamelanotide for more than 2 years. They completed the EQ‑5D‑3L and EPP‑QoL for their current health status, and retrospectively for a phototoxic reaction and treatment interruption. The IPPN highlighted that people in this study were on long-term treatment (more than 2 years) with afamelanotide had a utility value comparable to the age-matched general population (0.965). In addition, the utility value for a phototoxic reaction (0.215) and for a treatment interruption (0.331) were similar to utility values seen in people with acute burn injuries and chronic neuropathic pain.

Informal patient survey

4.22 The IPPN informal patient survey included 18 patients with EPP: 13 had been on afamelanotide for more than 2 years and 4 for less than 2 years. One of them had not had any treatment. The utility value for people who had been on treatment for more than 2 years was similar to the feasibility study (0.975), but lower for people on treatment for less than 2 years (0.619). The survey estimated a utility value of 0.397 for no treatment. The committee recalled that it had heard from clinical and patient experts that time would be needed for the treatment effect of afamelanotide to be realised because people with EPP adapt and adjust their behaviour to avoid light. It considered that this may account for the difference in utility between people with more than 2 years of treatment and those with less than 2 years. The ERG explained that both of these studies from the IPPN were subject to high levels of uncertainty. This was because the sample sizes were small and limited information was provided on the methods used. It was not clear how people were selected for inclusion, and the results may be subject to recall bias. The ERG said it was also not clear what time of the year the questionnaires were completed, or how EQ‑5D values varied over the year. It noted that the German EQ‑5D tariff was used. The ERG explained that the studies indicate that the EQ‑5D may be appropriate for detecting QoL changes in EPP because they showed that each of its 5 domains was affected by the condition. The IPPN considered that this new evidence addressed the uncertainty in the analysis about quantifying the quality-adjusted life year (QALY) gain associated with afamelanotide. But the committee considered that the IPPN's evidence was associated with substantial uncertainty. It welcomed the studies' submission and agreed that it would consider this evidence to aid its decision making as a part of a reasonable adjustment given the nature of the condition (see section 4.8).

4.23 In the second consultation, the IPPN also responded to the approach taken in this evaluation. It commented that the time horizon used in the modelling was 35 years, which was not a lifetime horizon. It said it believed a longer time horizon was more appropriate and in line with previous highly specialised technology evaluations. The IPPN noted that the incremental QALY gain estimated by the ERG's analysis was smaller than the utility benefit associated with oral administration that had been accepted by the committee for a previous evaluation (NICE's highly specialised technologies guidance on eliglustat for treating type 1 Gaucher disease). It highlighted that this did not appear to be valid. The committee acknowledged that the IPPN's comments were in line with its own concerns that the ERG's analysis may not have captured all the benefits associated with afamelanotide treatment.

Nature of the qualitative evidence

4.24 The committee discussed at length the nature of the patient's and carer's testimonies as evidence. It was also aware of the NICE real world evidence framework which lays out general principles for generation of real-world evidence prospectively. The committee considered that the existing observational data is subject to various biases, and that this should be taken into account when interpreting the data and assessing its contribution to decision making. It also highlighted that it considers qualitative evidence as part of its careful deliberation on all the factors that have contributed to its conclusion. For example, it contributes to the understanding of the nature of the condition, and to interpreting the clinical evidence. The committee agreed that qualitative evidence collected systematically and analysed using standard qualitative techniques could potentially have provided more scientifically robust information on the full breadth of patient experiences. It recognised that, in that sense, the qualitative information it had been presented with had some limitations. However, it concluded that it was highly valuable in informing the nature of the condition, the benefits of the treatment and the meaning of those benefits for people with the condition and their families. Given the specific challenges associated with EPP, the committee concluded that it was important to take into account patient testimonies and other qualitative evidence as part of its decision making.

4.25 It noted that relevant qualitative descriptions of health states associated with EPP could be used to elicit quantitative values (such as in a vignette study), which could be used in an economic model. It also noted that it was important to consider how the benefits of afamelanotide could be quantified as part of its decision making. The committee was also aware that the decision support unit had issued updated guidance on the use of qualitative evidence to inform generation of utility values in health technology assessment. It concluded that qualitative descriptions in the form of a vignette study could be used to inform the economic model.

Clinical-effectiveness results

4.26 The clinical trial results showed a statistically significant increase with afamelanotide compared with placebo in the median amount of time a person could spend in daylight without pain (CUV029, between 10:00 and 15:00: 5.63 hours with afamelanotide and 0.75 hours with placebo, p=0.006; CUV039, between 10:00 and 18:00: 69.4 hours and 40.8 hours respectively, p=0.044). For CUV039, this equates to an average of 23.1 minutes per day in daylight for people having afamelanotide, compared with 13.6 minutes per day for people having placebo, between 10:00 and 18:00; for context, the committee understood that healthy indoor workers spend an average of 22 minutes outdoors between 10:00 and 15:00 on summer weekdays. CUV029 also showed a statistically significant decrease in the number and severity of phototoxic reactions (77 reactions with afamelanotide and 146 with placebo, p=0.04; the data on severity is not reported because the company deemed them to be commercial in confidence). The clinical expert also described evidence on maximal light exposure durations (from an observational study), which further illustrated the size of benefits associated with afamelanotide. The patient experts and the British Porphyria Association (BPA) explained that being able to spend an extra few minutes in daylight or having fewer phototoxic reactions could have a large effect on people's lives. For example, a few minutes might allow a person with EPP to get into a shop or travel to work. A patient expert also explained that a few minutes in full daylight would typically equate to many more minutes, and even hours, in dappled light or shade. This would mean people with EPP would be in a much stronger position to manage their lives without being debilitated by the disease. The comments received following consultation strongly echoed these statements. Furthermore, in their testimonies, patients reported that afamelanotide resulted in much better outcomes than it had in the clinical trials. For example, a patient expert at the meeting stated that afamelanotide had allowed him to increase the time he spent in light by hours rather than by minutes (as had been seen in the trials), and described this as life changing.

4.27 In 1 long-term observational study (Biolcati et al. 2015), QoL scores measured by the EPP‑QoL increased from 32% to 74% of the maximum in the first 6 months of afamelanotide treatment. A clinical expert stated that this increase in the first 6 months was important. The committee also acknowledged that DLQI data from the clinical trials had shown a non-statistically significant improvement in QoL with afamelanotide. However, the committee also noted that there was little further change in QoL (EPP‑QoL) over the following 6 years in the observational study. This indicated that there was no marked improvement in QoL as measured by EPP‑QoL in patients who had treatment beyond the duration of the controlled clinical trials. The clinical expert speculated that the climate in Switzerland and Italy may have contributed towards the stabilisation in scores beyond 6 months. The committee considered that these results were in contrast to the discussions around the slow resolution of conditioned light avoidance. The committee also noted that there was an improvement in QoL scores in the placebo arm of the clinical trials. The company explained that this was likely because EPP is a neglected disorder and the opportunity to enrol in a trial would have provided patients hope for the first time. The committee recalled the testimony from clinical and patient experts highlighting the impact of the treatment benefits on QoL (see section 4.26). The committee concluded that afamelanotide was likely to improve QoL, but the true size of any improvement was uncertain.

4.28 The company and experts stated that another indicator of the effectiveness of afamelanotide was the adherence rate in 2018 of 98.5%, despite the cost and time associated with travel for treatment. A patient organisation also highlighted a survey showing that 93% of people surveyed would want to try afamelanotide. The committee appreciated that the adherence rate was high, but noted that it was not a direct marker of effectiveness and did not quantify the size of the treatment benefit.

4.29 The committee asked if there was any evidence about how the severity of EPP affected outcomes with afamelanotide, and heard that there was no specific data on this. However, the clinical experts suggested that, anecdotally, afamelanotide had been effective across the whole trial population.

4.30 After the second evaluation consultation, the committee considered some more recently published studies, which were highlighted by stakeholders:

4.31 The patient experts said the recently published studies provided additional evidence to quantify the benefits of afamelanotide treatment. They also said that they did not think recall bias concerns in the studies were necessarily valid because people with EPP can usually recall events such as a phototoxic reaction accurately. The committee acknowledge that the recently published evidence provided more information on the treatment effects of afamelanotide. However, it noted the limitations of the data outlined by the ERG, and was aware that none of the data provided could be used to inform the economic model. The committee concluded that it would take the study results into account in its decision making.

Clinical-effectiveness conclusions

4.32 The committee considered the breadth of the clinical-effectiveness evidence, including the clinical trials, observational studies, stakeholder-submitted evidence and testimonies. It acknowledged the challenges in measuring the benefits of afamelanotide, and that these influenced the limitations in the clinical-effectiveness evidence for the drug. It also accepted that the clinical trial results may not have adequately represented the benefits of afamelanotide, and highlighted the influence of qualitative evidence, in particular, to help further clarify the clinical significance and practical meaning of the treatment effect for patients and their families. The committee noted the benefits associated with afamelanotide in clinical trials, and recalled the importance and value of those benefits for patients and their families. Nevertheless, the committee considered it important to take into account the uncertainties in the evidence and in the quantification of the benefits. It considered that, although it believed that afamelanotide did offer a clinical benefit, the size of the benefit remained uncertain. Overall, the committee concluded that afamelanotide is effective and provides important benefits for patients, but further concluded that there are important uncertainties in the evidence and the size of the clinical benefits due to the challenges specifically associated with collecting data in EPP.

Company's model

4.33 A large amount of information relating to the company's model structure and assumptions was considered confidential by the company. The committee was disappointed because this meant that its discussions and decisions on the model could not be fully described publicly. The company's analyses estimated disability-adjusted life years (DALYs) averted, and the incremental cost-effectiveness ratios (ICERs) were presented as cost per DALY averted. The company's model stratified the condition into mild, moderate and severe disease, based on EPP‑QoL scores divided into 3 equal ranges (that is, 67 to 100, 33 to 67 and 0 to 33 respectively), and each range was assigned a disability weight to generate DALYs. The effectiveness of afamelanotide and standard care was based on the proportion of people in the 3 ranges, using pooled clinical trial data on the EPP‑QoL collected at 4 months. The ERG presented several exploratory analyses, including direct conversion of the company's approach to generate QALYs instead of DALYs and an alternative modelling approach.

4.34 The committee discussed the company's stratification of the condition into mild, moderate and severe disease. The company considered that arbitrary division of the EPP‑QoL into thirds to be the fairest approach in the absence of validated cut-offs for EPP severity using the EPP‑QoL. The committee recalled the challenges associated with measuring QoL in EPP using EPP‑QoL (see section 4.18). It concluded that the company's approach to stratifying disease severity according to arbitrary quantiles contributed to the uncertainties in the economic modelling.

Disability-adjusted life year framework

4.35 The company stated that it did not support using utility values to quantify QoL in EPP because of the unique nature of the condition and because there was of a lack of available robust data from which to derive utility values. Rather, the company considered that it was more appropriate to consider the impact of EPP and afamelanotide on people's QoL in terms of disability. At the second evaluation committee meeting, the company stated that it did not consider the DALY approach to be more appropriate than QALYs. It considered that no approach entirely reflected the complexities in EPP, and that the DALY model was its attempt to present an alternative approach. The committee noted that the NICE interim process and methods guide of the Highly Specialised Technologies Programme states that, as part of the consideration of value for money, the committee will consider the ICER expressed as an incremental cost per QALY gained. It stated that using QALYs was in the NICE reference case (the preferred methods to be applied consistently across evaluations), and that this was important to allow consistent evaluation across therapy areas. The committee was aware of the importance of the consistent approach used by NICE and the NHS to ensure fair allocation of finite budgets because funding of a treatment may mean other treatments or services are displaced. The committee can, however, consider non-reference case methods alongside those in the reference case if there is a strong enough case for it. However, the committee was not persuaded by the argument for preferring an analysis based on the DALY rather than the QALY. The committee was aware that the ERG had provided a simple adaptation of the company's model, which showed that the DALY and the QALY approaches produced similar ICERs. The committee concluded that, although it would take the DALY-based model into account in its decision making, its preferred approach was the one aligned with the NICE reference case.

Generating DALYs and QALYs

4.36 In its DALY-based framework, the company used disability weights from the World Health Organization Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) to model the disability associated with mild, moderate or severe EPP. However, because the GBD survey did not ask about EPP, the company used weights for a proxy condition it considered similar to EPP in its modelling. The company considered the proxy condition to be confidential. The committee appreciated similarities between some important aspects of the EPP and the proxy condition, but was aware of other important aspects that were quite different. It was unclear about the extent to which the proxy condition reflected the disability associated with EPP, and whether it was valid to assume that the disability associated with mild, moderate or severe disease in the proxy condition would correspond with mild, moderate or severe EPP. Furthermore, it reiterated its concerns about the uncertainties surrounding the stratification of people with mild, moderate and severe EPP based on EPP‑QoL data collected in the trials (see section 4.18). The committee questioned further why the company preferred to map from other diseases that may not be fully representative of EPP rather than directly using patient-level QoL data collected in EPP trials. The committee understood from the company that it needed a proxy condition to derive disability weights because these were not available for EPP. However, it did not consider that the company had made a strong case for using disability weights to justify the added uncertainty of using a proxy condition rather than direct trial data. The committee concluded that the proxy condition used by the company may not fully capture the experience of people with EPP, and the assumption that it is similar to EPP in general and at different levels of severity was not sufficiently robust.

4.37 The committee queried whether it would be possible to generate utility weights (and hence QALYs) from the EPP‑QoL data (for example, by mapping to EQ‑5D). It noted that EPP‑QoL has not been assigned preference weights or mapped to a tool such as EQ‑5D. The committee understood that it was not possible to generate utility weights directly from the EPP‑QoL data. After the committee meeting, 1 clinical expert suggested that EPP‑QoL visual analogue scale (VAS) data could be used to infer utility values. However, the ERG highlighted that there were several problems in using VAS data in this way. In particular, it noted that, fundamentally, VAS data represented health status and not utility.

4.38 The ERG presented alternative methods to generate QALYs in exploratory analyses. It presented 2 analyses in which the company's modelling approach was directly converted to generate QALYs. One approach involved converting the disability weights to utilities, the other involved by sourcing utilities rather than disability weights for the same proxy condition from a published source. These approaches both used the same disease stratification and proxy condition as the company's base case. The ERG also presented an alternative approach in its exploratory base case, in which it used DLQI data for afamelanotide and standard care from 1 clinical trial (in place of the disease stratification approach in the company's model). The DLQI scores were mapped to EQ‑5D to derive utility values using a published, validated algorithm. The committee considered that this approach provided a more direct link between QoL measured in the clinical trials and the modelled benefits, with fewer assumptions than the company's proxy condition base-case approach. However, it reiterated questions about whether the DLQI measured in the trials adequately captured the QoL associated with EPP and the benefits of afamelanotide (see section 4.19). The committee therefore considered that the ERG's approach may have underestimated the real-life benefits of afamelanotide because these may have been underestimated in the trials and may not have been captured in the DLQI. However, the committee noted that it was not possible to quantify the underestimation.

4.39 Overall, the committee acknowledged important limitations in all the presented modelling approaches. It considered that quantifying the effects of the condition and benefits associated with afamelanotide, and translating those into QALYs, was a crucial uncertainty in the economic modelling. It highlighted that, even if the EPP‑QoL is the best available tool for capturing the burden of EPP it still had uncertainties (see section 4.18), and the modelling based on EPP‑QoL still relied on the disease stratification and proxy condition. The committee recognised that the appropriateness of DLQI and its sensitivity to the effects of EPP and treatment have been questioned. However, it acknowledged that it would capture some relevant aspects (see section 4.19), and that the modelling based on it provided a more direct link between QoL measured in the clinical trials and the modelled benefits. The committee concluded that on balance all the modelling approaches had limitations.

Additional assumptions

4.40 The committee noted that the modelling was based on EPP‑QoL data collected at 4 months, but that this data was also collected at 6 months, although from a smaller proportion of the trial population. This data had not been presented by the company. The committee considered that, if the EPP‑QoL data was to be used, the longer follow-up data could have been useful to see. This was particularly because 1 clinical expert explained that the benefits of afamelanotide may take time to become apparent if people adapt their conditioned behaviour gradually.

4.41 The company assumed in its modelling that the benefits of afamelanotide would be immediate and would remain constant for the whole year, including after the last implant. The ERG tested some assumptions around this in sensitivity analyses. These included analyses around how long it would take for a person to experience the benefits of afamelanotide, and how long the treatment effects of afamelanotide would persist after the last implant of the year. The committee considered that it was likely that it would take some time before patients would experience the benefits of afamelanotide, not least because time would be needed to unlearn conditioned behaviour associated with light avoidance. The clinical experts described how the protective antioxidant effect of afamelanotide needed time to build up after the first implant, but would persist for a period of time after the last implant. The committee noted the lack of data to support these assumptions. However, on balance, it concluded that the ERG's analyses assuming that the effect of afamelanotide would build up over the first 2 months (as modelled in its base case), and that the treatment effect would slowly decrease over 6 months after the last implant, used plausible assumptions.

4.42 The committee discussed the likely dosage of afamelanotide in clinical practice. It was aware that the marketing authorisation recommended administering an implant every 2 months before expected, and during increased, sunlight exposure from spring to early autumn, and recommended a maximum of 4 implants per year. The clinical experts stated that they expected the implants to be used from around March to October in England, meaning that 4 implants would often be used, but that some people may not need the maximum number. The company provided an estimate of the average number of implants people with EPP may have. This was based on what had been seen in expanded access and commercial distribution of the drug across the expected EPP population. This number is not reported because the company deemed it to be commercial in confidence. The company did not provide any detail on whether its estimate was generalisable to people using afamelanotide in clinical practice in England. The committee concluded that it should take into account that people may have up to 4 implants per year in its decision making, and noted that the data from the Wensink et al. (2020) study supported this.

Role of ICERs in decision making

4.43 The company challenged the use of the ICER in the committee's decision. It, and other stakeholders, considered that the ICER alone had determined the initial decisions, and that inherent problems had not been taken into account. In particular, the company emphasised the significant challenges in measuring the effects of EPP and the benefits of afamelanotide, and considered that the QALY estimates – and hence the ICERs – were therefore not appropriate in this case. The committee was mindful of its remit (that is, to consider the clinical effectiveness and value for money of technologies) and the crucial importance of considering value for money in a fair and consistent way as part of this remit. It emphasised that an important part of NICE's approach to achieving a fair and consistent health technology evaluation programme was to use QALYs as a measure of treatment benefit. The committee emphasised that value for money must remain an important (but not the only) part of the decision in this case, and that the ICER was not the only contributor to its view on value for money. It explained that all its decisions on highly specialised technologies are reached by in-depth committee deliberation. The committee considered that the methods for establishing ICERs in this population were not so uncertain as to be unreasonable for them to contribute to decision making, provided those ICERs are considered in the context of the associated challenges, limitations and uncertainties. It concluded that it was appropriate to consider the ICERs for afamelanotide as part of its consideration of value for money.

QALY weight

4.44 The committee understood that the interim process and methods of the highly specialised technologies programme (2017) specifies that a most plausible ICER of below £100,000 per QALY gained for a highly specialised technology is normally considered an effective use of NHS resources. For a most plausible ICER above £100,000 per QALY gained, judgements about the acceptability of the highly specialised technology as an effective use of NHS resources must take account of the size of the incremental therapeutic improvement, as revealed through the number of additional QALYs gained. For a QALY weight to be applied, there will need to be compelling evidence that the treatment offers significant QALY gains. The committee discussed the QALY gains associated with afamelanotide. The QALY gains were driven by improvements in QoL. The committee considered the size of the treatment benefits predicted by the models in both the company's base case and ERG's exploratory analyses. The undiscounted incremental DALYs in the company's base case and the ERG's estimated incremental QALYs based on the company's use of a proxy disease cannot be reported because the company has stated that these are commercial in confidence. Over the lifetime of a patient, the undiscounted QALYs gained with afamelanotide in the ERG's exploratory base case were 0.56, and did not exceed 0.8 in the ERG's sensitivity analyses. The committee noted that the ERG's analysis may have underestimated the QALY gains associated with afamelanotide. The committee recalled that there was uncertainty around the utility estimates (and the disability estimates in the company's model), and also recalled that the quantification of the benefits of afamelanotide was a critical uncertainty in this evaluation. Taking into account the clinical trial results, observational studies and qualitative evidence including patient and clinician testimonies, it recognised that the economic modelling was likely to have fallen short in capturing the full benefits of treatment in the QALY estimates. The committee considered the undiscounted QALY gains from the updated analysis after the second consultation, which included the QoL data submitted by the IPPN. The committee recalled that this evidence was subject to high levels of uncertainty, which it believed could not be used to provide compelling evidence of an incremental QALY gain of at least 10. The committee concluded that the criteria for applying a QALY weight were not met.

Cost-effectiveness results: ICERs

4.45 The committee considered the results of the ERG's exploratory base case and noted that the following ICERs were all over £100,000 per QALY gained:

The committee noted that the ICER was £1,726,802 per QALY gained using the EPP‑QoL data and disease stratification, and the company's preferred proxy condition (based on utilities from the literature rather than disability weights). It considered this to be very similar to the ERG's exploratory base-case ICERs. The committee preferred the assumptions used by the ERG to those used by the company. However, it concluded that the ICERs based on the ERG's exploratory base case were unlikely to be plausible. This was because the base case may have underestimated the QALY gains associated with afamelanotide.

Cost-effectiveness scenarios after the second consultation

4.46 After the second consultation the committee considered further exploratory cost-effectiveness scenarios based on the new QoL data submitted by stakeholders (see section 4.20). The following exploratory scenarios were considered by the committee:

Challenges in quantifying treatment benefits in the economic modelling

4.47 The committee was aware that a critical uncertainty for the evaluation was in the quantification of treatment benefits and the translation of the benefits into QALYs (see section 4.39). The committee recalled its consideration of the importance of value for money as part of its remit (see section 4.43). It also recalled that, taking into account the clinical trial results, observational studies and qualitative evidence, it recognised that the economic evaluations were likely to have fallen short in capturing the full benefits of treatment in the QALYs (see section 4.43). The committee recognised that some of the problems with the economic modelling were influenced by the challenges of measuring the effects of the condition and treatment. It considered that it was appropriate to make a decision based on the available evidence (while taking into account its limitations). Given the nature of EPP, it also considered that it would be reasonable to consider alternative methods to capture the benefits of afamelanotide.

4.48 Before the second consultation the committee explored ways to quantify the health benefits described by patients' and clinical experts' testimonies in terms of QALYs. It suggested that utility scores for the economic model could be estimated through an indirect method such as a 'vignette' study. Such a study would collect patient or expert experiences to form a detailed, qualitative description of each disease health state (a 'vignette'). The QoL associated with each vignette could then be quantified, using established methods, preferably by the general population or alternatively by clinical experts, to provide an objective estimate of utility. The committee suggested that such a study should consider the effects of all aspects of the condition and the treatment on health-related QoL, including the totality of the patient experience and, for example, effects of employment and impacts on the health of family members. The committee explained that similar approaches had previously been considered in other highly specialised technologies evaluations when direct measurement was not possible. It acknowledged that such approaches are not necessarily as robust as the preferred approaches specified in the NICE reference case, but that it would be reasonable to consider given the challenges associated with this condition. The committee considered that, if such a study was submitted, it may be possible to refine the QALY estimates and then reconsider with a higher degree of certainty the QALY gains and value for money of afamelanotide.

4.49 After the second consultation, the committee was disappointed that the company had chosen not to do a vignette study. The committee took into account responses from other stakeholders and new QoL evidence they submitted (see section 4.20). Although the new evidence was associated with high levels of uncertainty, the committee considered that it added to the evidence base. It provided further evidence on QoL in EPP and its response to afamelanotide, in addition to that estimated from the trial data. The committee recalled that it considered the clinical trial evidence may have underestimated the treatment benefit of afamelanotide. It also recalled that, because of the nature of EPP, collecting health-related QoL data for people with the condition is challenging. The committee agreed that the new QoL evidence submitted by the patient groups provided more information on the experiences of people with EPP. However, the evidence was still highly uncertain.