Article Text
PDF
Abstract
Background/aims To examine the agreement in clinical decisions of glaucoma status made in a virtual glaucoma clinic with those made during a face-to-face consultation.
Methods A trained nurse and technicians entered data prospectively for 204 patients into a proforma. A subsequent face-to-face clinical assessment was completed by either a glaucoma consultant or fellow. Proformas were reviewed remotely by one of two additional glaucoma consultants, and 12 months later, by the clinicians who had undertaken the original clinical examination. The interobserver and intraobserver decision-making agreements of virtual assessment versus standard care were calculated.
Results We identified adverse disagreement between face-to-face and virtual review in 7/204 (3.4%, 95% CI 0.9% to 5.9%) patients, where virtual review failed to predict a need to accelerated follow-up identified in face-to-face review. Misclassification events were rare, occurring in 1.9% (95% CI 0.3% to 3.8%) of assessments. Interobserver κ (95% CI) showed only fair agreement (0.24 (0.04 to 0.43)); this improved to moderate agreement when only consultant decisions were compared against each other (κ=0.41 (0.16 to 0.65)). The intraobserver agreement κ (95% CI) for the consultant was 0.274 (0.073 to 0.476), and that for the fellow was 0.264 (0.031 to 0.497).
Conclusions The low rate of adverse misclassification, combined with the slowly progressive nature of most glaucoma, and the fact that patients will all be regularly reassessed, suggests that virtual clinics offer a safe, logistically viable option for selected patients with glaucoma.
- Diagnostic tests/Investigation
- Glaucoma
- Telemedicine
Statistics from Altmetric.com
Introduction
Chronic disease management is a particular challenge facing the UK National Health Service (NHS).1 It is estimated that the UK population is growing at a rate of 0.7% per annum, and will have reached approximately 72 million by 2031. As with most Western countries, the UK has an increasingly ageing population; in 2003, there were 20 million individuals over the age of 50 years, and this number is predicted to increase by 36% by 2031.2 Older people are disproportionately heavy users of healthcare services, with implications for both resource funding and management.3 The NHS is also under increasing pressure to make efficiency savings.4 ,5
Current models of care for many chronic diseases require that patients make repeated visits to hospitals and clinics in order to undergo assessments and tests. For an efficient, consultant-led service, clinically relevant information should be presented in such a way to allow rapid decision-making. Stable patients are then quickly identified, allowing consultants more time to discuss difficult management decisions with unstable patients. This requires the segregation of patients with the same disease process into different categories depending on clinical risk.
Glaucoma, as an example of a chronic eye condition, is the most common cause of irreversible blindness worldwide,6 but blindness can be prevented by early diagnosis and treatment.7 Chronic diseases often require lifelong monitoring with periodic treatment adjustment. It has been estimated that in England around half a million people suffer with glaucoma, with up to 1.2 million suffering from ocular hypertension or suspected glaucoma, which approximates to 2.4 million NHS outpatient visits per year.8 There are increasing capacity problems coping with the number of glaucoma-related outpatient appointments within the NHS hospital eye service (HES). Modelling predicts that this is set to worsen with the current methods of NHS glaucoma patient management.9
It has been suggested that a ‘virtual clinic’ model of community-based monitoring of stable glaucoma disease may help tackle capacity issues in the UK HES. For example, clinical test data can be collected by technical staff, uploaded to a server and presented to a consultant ophthalmologist logging on remotely.
While there are now a number of ophthalmology services across the UK that provide versions of these ‘virtual clinic’ services, there is currently no evidence that this method of clinical management is at least equivalent to face-to-face consultations in safety and robustness of decision-making. The purpose of our study was to examine the agreement between the decisions made by clinicians reviewing patient data remotely with those made by clinicians undertaking a face-to-face consultation (ie, standard outpatient care). There were two research objectives: first, to establish the interobserver agreement between remote review clinicians and those delivering standard outpatient care, and second to evaluate the intraobserver agreement, by comparing the decisions made by clinicians who provided standard care with the decisions they made when reviewing patient data remotely.
Methods
Consecutive adult patients with glaucoma with a planned, greater than six monthly follow-up frequency were considered by the Glaucoma Service to be suitable for this type of clinic. Patients with poor mobility, poor visual field technique or poor-quality optic disc imaging or other diagnosis not suited to remote assessments were not considered suitable. Only patients with open angles (including pseudoexfoliation and pigment dispersion syndrome) were considered. The hospital's performance and information team provided the details of patients' follow-up schedules and over a 7-month period from March to September 2011 for two adult glaucoma outpatient clinics.
The ‘virtual clinic’ was set up in the hospital's clinical research facility and was staffed by three technicians and an ophthalmic nurse. Patients underwent the same routine investigations and examinations that were to be conducted in their clinic appointment. The flow of the clinic is as follows: each technician manned a designated workstation––visual acuity measurement, visual field measurement (Humphrey Field Analyzer, Zeiss, California, USA), optic disc imaging. The latter consisted of disc and macular photography (TRC NW8, Topcon Medical Systems, Oakland, USA) and scanning laser ophthalmoscopy with the Heidelberg Retina Tomograph (HRT; Heidelberg Engineering, Heidelberg, Germany). A questionnaire was used to capture aspects of the patient's symptoms and history, which was reviewed by the nurse. The nurse also measured and recorded intraocular pressure (IOP) data using a Goldmann Applanation Tonometer and made a slit-lamp assessment of the anterior segment. The nurse then completed a data-capture proforma that contained aspects of the clinical examination from that day, including any comment on the patient's symptoms where relevant. Advice was offered by the nurse on common anterior segment complaints, such as blepharitis. Significant drop-related side effects were recorded. The nurse also observed drop technique and offered advice where appropriate. The patient then went on to see a consultant (JC) or senior fellow (RP) for their face-to-face consultation. The clinicians' observations of the patient's glaucoma status and change in management were recorded. The follow-up outcomes were subsequently risk-stratified to best fit the categories required in the remote reviews and shown in table 1.
Risk stratification for remote assessors
The audit department authorised the study team (audit code: CA15/GL/25) to collect data on the patients for remote assessment by members, as described below.
Patient details including ethnicity, diagnosis, general health status, angle assessment, central corneal thickness measurement, maximum IOP over the course of their follow-up in the service, results of any phasing examinations and drop usage were recorded on a separate paper proforma, along with a summary of the clinical assessment at their preceding outpatient appointment. These data, along with all current and previous visual field plots and current HRT results (including trend analysis where available), were available for ‘remote review’. Consultants undertaking the remote review (PJF and KB) also had access to digital optic disc photographs.
Remote reviewers were asked to indicate when the next follow-up visit should be, based on the patient's risk of developing significant visual loss over the follow-up interval (table 1). They were also advised that they could comment upon the glaucoma status and whether they noted any other factor in the examination that warranted further investigation.
Twelve months after the patients' visits, the two clinicians who conducted the face-to-face assessment (JC and RP) were asked to perform the remote review process in order to determine whether their management decisions would be the same as that made when they undertook the face-to-face assessment. The time frame was an attempt to prevent the clinicians from recalling their original decisions.
Following collection of data, the characteristics of the clinical findings that led to the clinical conclusion of progressive change were presented to the coauthors. Agreement between the authors that there was evidence of disease progression was then achieved before data analysis was completed.
Data analysis
Cohen's κ statistic was used to assess both interobserver (ie, face-to-face clinicians' vs remote review clinicians') agreement and intraobserver (same clinicians' face-to-face vs their deferred/later remote review) agreement.
Using the face-to-face review as the reference standard, the sensitivity and specificity of the ability of the virtual clinic assessment to detect unstable disease were also calculated. We also looked at the agreement between specific clinicians.
Results
Two hundred and eighty-six patients reviewed over a 6-month period and screened for eligibility for inclusion, with 217 meeting the criteria for having their data remotely reviewed. Data for 13 patients were irrevocably damaged following a flooding incident; thus, complete data were available for 204 patients.
The primary diagnoses were: primary open-angle glaucoma (POAG) n=88 (43%), primary ocular hypertension n=59 (29%), glaucoma suspects n=33 (16%), secondary glaucoma n=18 (9%), secondary ocular hypertension (OHT) n=4 (2%) and primary angle closure n=2 (1%). The average (SD; range) visual field mean deviation in the better and worse eyes were −1.63 (2.66; −15.14 to 3.35) dB and −4.96 (5.84; −27.94 to 1.99) dB, respectively.
At the face-to-face assessment, 21 (10%) patients were identified as being ‘unstable’, in that they required a change in management. In the virtual clinic assessment, patients deemed as ‘very low’ or ‘low risk’ were classed as ‘stable’, while those deemed ‘medium–high risk’ or ‘emergency’ were recorded as ‘unstable’. Where the reviewer had indicated that the classification was the result of a non-glaucomatous issue, and that the glaucoma itself was stable, patients were marked as ‘stable’.
Interobserver agreement
All five authors reviewed the hospital medical records for patients considered unstable on face-to-face assessment, and stable on virtual assessment (N=14). Thus, consensus agreement was ensured for the face-to-face clinical suspicion of ‘unstable’ glaucoma or OHT. This consensus opinion led to seven patients deemed ‘unstable’ at their face-to-face review as being reclassified ‘stable’. The κ table is shown in table 2. The κ statistic for the interobserver agreement, comparing face-to-face with virtual assessment, was 0.320 (95% CI 0.112 to 0.529), indicating ‘fair agreement’. The sensitivity and specificity (95% CI) of the virtual clinic were 50.0 (23.0 to 77.0)% and 91.6 (86.7 to 95.1)%, respectively.
κ table for interobserver agreement analysis
Furthermore, we examined the agreement between consultants only, excluding any data from senior trainees (fellows). Table 3 illustrates the agreement between consultant face-to-face review and remote assessment. This suggests that consultant virtual clinic decisions agree well with consultant face-to-face management decisions.
Agreement, sensitivity and specificity between virtual clinic decisions and clinical management decisions made by consultant
The 7/204 (3.4%, 95% CI 0.9% to 5.9%) patients who were ‘misclassified’ as stable during the virtual clinic assessment, but ‘unstable’ at the face-to-face review, comprised of one patient with POAG, two glaucoma suspects and four patients with glaucoma secondary to either pseudoexfoliation or pigment dispersion syndrome. Further details of their diagnoses and visual status are reported in table 4. The characteristics suggest that five of the patients had ‘early’ disease based on the depth of their visual field defect.10 The other two with more advanced disease were recommended to reattend in 6 months in the virtual clinic assessment.
Characteristics of patients who were not identified as progressing at the remote review
Intraobserver agreement
The consultant and fellow undertaking clinical examinations performed ‘virtual clinic’ assessments for 194 of the original 204 patients. The intraobserver agreement κ (95% CI) for the consultant was 0.274 (0.073 to 0.476), and for the fellow was 0.264 (0.031 to 0.497). The sensitivity (95% CI) and specificity (95% CI) of ‘virtual clinic’ management decisions were 75.0 (34.9 to 96.8) % and 81.0 (72.1 to 88.0) % for the consultant and 60.0 (26.2 to 87.8) % and 78.9 (67.6 to 87.7) % for the fellow, respectively.
In total, six patients were deemed as ‘stable’ during the virtual clinic assessment (four by the fellow and two by the consultant), and were deemed unstable during the face-to-face examination. During the remote review, the disc imaging of the two patients progressing in the consultants' clinical assessments was not available.
As expected, copathology was identified in the clinical assessment. Potentially significant symptoms were successfully identified during the remote assessment, but were rare (diplopia 1%, amaurosis fugax 0.5%, photopsia 0.5%). Appropriate, additional investigations were recommended for all of these patients. One (0.5%) leaking trabeculectomy bleb was identified clinically, but not via the remote assessment. Common, non-sight threatening anterior segment disorders (episcleritis, blepharitis, drop allergy) that required additional treatment as a result of the clinical assessment were identified remotely in only one of six (16.6%) patients, through the patient questionnaire. New retinal findings (dry age related macular degeneration, epiretinal membrane) were identified in four (2%) of patients on clinical assessment, and half of these were identified with the remote assessment from fundus photography.
Discussion
Our study aimed to evaluate the agreement between ophthalmologists' management decisions made during the standard outpatient visit and a ‘virtual clinic’. Intraobserver agreement for both consultant and fellow grades were similar (0.27 vs 0.26) and only moderate, and the classification categories may be too arbitrary, particularly with the separation of 6 and 12 months into low and very low risk. The results suggest that the virtual clinic has low sensitivity at detecting unstable disease. However, closer inspection shows that agreement and sensitivity are dependent on the grade of the clinician, with non-consultants adopting a more cautious and less consistent approach in their management decisions. Decision-making between remote and face-to-face assessments are likely to differ, but the identification of progressing disease is critical. The wide range in CIs for sensitivity suggest that far larger and long-term studies are required to confirm virtual clinic sensitivity for detecting progressing disease. The number of progressing patients is expected to be small in this population of patients.
A similar study was conducted by Gupta et al11 in a tertiary ophthalmic referral centre in India. Agreement was assessed between ophthalmologists diagnosing and managing patients during their clinical examination with those made via teleophthalmology review. The κ statistic for the diagnosis and management of glaucoma cases was 0.52 and 0.53, respectively. Another study examining the agreement between a district hospital ophthalmologist and a specialist glaucomatologist undertaking teleophthalmology reviews found similar levels of agreement in diagnosing glaucoma in African eyes.12 The relatively low level of agreement in management decisions was not unusual. A single ophthalmologist showed some disagreement with their own management decisions when presented with cases a period after their initial clinical examination.13 There exists a diversity of clinical opinion among glaucomatologists on the best management options for individual patients.14
Decisions based on multiple investigations, all open to interpretation, are likely to lead to variations in recommended outcomes. In our study, the more experienced clinicians were prepared to accept more uncertainty and recommend longer follow-up periods. The study design forced clinicians to classify the patient's risk of glaucoma progression. The arbitrary distinction between stable (≥6 months follow-up) and unstable (<6 months follow-up) may have led to over-representation of poor stability identified in the study (table 2).
All of the patients remained under the hospital’s care. This continuation of care would give an opportunity for ‘instability’ to be detected at subsequent visits. Glaucoma progression can be difficult to detect, particularly if based on clinician judgement of paper-based visual field printouts.15 With the inherent noise within visual field test results, clinicians may reasonably adopt a ‘wait and see’ approach to managing suspected disease progression.16 In this study, the two patients (2/204, 1.9%, 95% CI 0.3% to 3.8%) in whom there was relatively advanced visual field loss and disagreement between clinical assessment and virtual assessment were recommended to be seen again in 6 months. Progression seen at that virtual assessment would instigate a change in management. Large-scale modelling studies have shown that patients with early disease, defined as less than -6 dB of mean deviation sensitivity loss,10 are at low risk of progressing to blindness once they are in the ophthalmology healthcare system.17 Thus, perhaps the traditional outpatient consultation should remain for patients with worse disease status, who are at a higher risk of developing significant visual loss over their lifetime.
While concerning symptoms, suggestive of new onset and non-glaucoma associated eye disease, were identified remotely, other slowly changing processes, such as age-related macular degeneration were not well recognised, and patients should be advised of the limitations and recommended to undertake regular community optometry assessments. Suitable correspondence between HES and community optometrists is important to clarify the limitations of virtual review clinics. Local side effects to treatment and common ocular surface diseases, such as blepharitis, are poorly identified on remote assessments, and a suitably trained ophthalmic nurse present at the data-gathering assessments will identify and manage most of these issues. The nurse can also make recommendations on the extent of visually significant lens opacity.
In summary, our study suggests that the use of a virtual clinic in the management of stable glaucoma patients has some limitations, and may not provide the level of sensitivity at detecting unstable disease that is found in the standard outpatient environment. However, our study identified no serious safety concerns, and suggests that significant misclassification events are rare, occurring between 0.3% and 3.8% of assessments. The slowly progressive nature of most glaucoma and the periodic reassessments suggest that virtual clinics offer a safe, logistically viable option for selected patients. We recommend patients enrolled in such schemes be those at low risk of progression to significant visual loss over each follow-up interval. Remote assessments are most reliably performed by consultant-level glaucoma specialist. Patients should be encouraged to see their optometrist routinely to assess for new, unrelated eye disease.
References
Footnotes
Contributors JC: Study design, data collection, data interpretation, manuscript preparation. RP: Study design, data collection, data interpretation, manuscript critique. AK: Data analysis, data interpretation, manuscript preparation. PJF: Study design, data collection, data interpretation, manuscript critique. KB: Study design, data collection, data interpretation, manuscript critique.
Funding The research was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology.
Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- At a glance