Which of the following refers to the type of potential problem that nurses manage using both independent and physician prescribed interventions?

Cochrane Database Syst Rev. 2018; 2018(7): CD001271.

Monitoring Editor: Miranda Laurant,

Radboud Institute for Health Sciences, IQ healthcare, Radboud University Medical Center, PO Box 9101, NijmegenNetherlands, 6500 HB

HAN University of Applied Sciences, Faculty of Health and Social Studies, NijmegenNetherlands

Zorgbelang Gelderland, ArnhemNetherlands

Faculty of Medicine, Siriraj Hospital, Mahidol University, Department of Obstetrics and Gynaecology, MahidolThailand

The University of Manchester, Centre for Health Informatics, Institute of Population Health, Williamson Building, 5th Floor, Oxford Road, ManchesterGreater ManchesterUK, M13 9PL

Abstract

Background

Current and expected problems such as ageing, increased prevalence of chronic conditions and multi‐morbidity, increased emphasis on healthy lifestyle and prevention, and substitution for care from hospitals by care provided in the community encourage countries worldwide to develop new models of primary care delivery. Owing to the fact that many tasks do not necessarily require the knowledge and skills of a doctor, interest in using nurses to expand the capacity of the primary care workforce is increasing. Substitution of nurses for doctors is one strategy used to improve access, efficiency, and quality of care. This is the first update of the Cochrane review published in 2005.

Objectives

Our aim was to investigate the impact of nurses working as substitutes for primary care doctors on:

• patient outcomes;

• processes of care; and

• utilisation, including volume and cost.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), part of the Cochrane Library (www.cochranelibrary.com), as well as MEDLINE, Ovid, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) and EbscoHost (searched 20.01.2015). We searched for grey literature in the Grey Literature Report and OpenGrey (21.02.2017), and we searched the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov trial registries (21.02.2017). We did a cited reference search for relevant studies (searched 27.01 2015) and checked reference lists of all included studies. We reran slightly revised strategies, limited to publication years between 2015 and 2017, for CENTRAL, MEDLINE, and CINAHL, in March 2017, and we have added one trial to ‘Studies awaiting classification’.

Selection criteria

Randomised trials evaluating the outcomes of nurses working as substitutes for doctors. The review is limited to primary healthcare services that provide first contact and ongoing care for patients with all types of health problems, excluding mental health problems. Studies which evaluated nurses supplementing the work of primary care doctors were excluded.

Data collection and analysis

Two review authors independently carried out data extraction and assessment of risk of bias of included studies. When feasible, we combined study results and determined an overall estimate of the effect. We evaluated other outcomes by completing a structured synthesis.

Main results

For this review, we identified 18 randomised trials evaluating the impact of nurses working as substitutes for doctors. One study was conducted in a middle‐income country, and all other studies in high‐income countries. The nursing level was often unclear or varied between and even within studies. The studies looked at nurses involved in first contact care (including urgent care), ongoing care for physical complaints, and follow‐up of patients with a particular chronic conditions such as diabetes. In many of the studies, nurses could get additional support or advice from a doctor. Nurse‐doctor substitution for preventive services and health education in primary care has been less well studied.

Study findings suggest that care delivered by nurses, compared to care delivered by doctors, probably generates similar or better health outcomes for a broad range of patient conditions (low‐ or moderate‐certainty evidence):

• Nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths (low‐certainty evidence).

• Blood pressure outcomes are probably slightly improved in nurse‐led primary care. Other clinical or health status outcomes are probably similar (moderate‐certainty evidence).

• Patient satisfaction is probably slightly higher in nurse‐led primary care (moderate‐certainty evidence). Quality of life may be slightly higher (low‐certainty evidence).

We are uncertain of the effects of nurse‐led care on process of care because the certainty of this evidence was assessed as very low.

The effect of nurse‐led care on utilisation of care is mixed and depends on the type of outcome. Consultations are probably longer in nurse‐led primary care (moderate‐certainty evidence), and numbers of attended return visits are slightly higher for nurses than for doctors (high‐certainty evidence). We found little or no difference between nurses and doctors in the number of prescriptions and attendance at accident and emergency units (high‐certainty evidence). There may be little or no difference in the number of tests and investigations, hospital referrals and hospital admissions between nurses and doctors (low‐certainty evidence).

We are uncertain of the effects of nurse‐led care on the costs of care because the certainty of this evidence was assessed as very low.

Authors' conclusions

This review shows that for some ongoing and urgent physical complaints and for chronic conditions, trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide equal or possibly even better quality of care compared to primary care doctors, and probably achieve equal or better health outcomes for patients. Nurses probably achieve higher levels of patient satisfaction, compared to primary care doctors. Furthermore, consultation length is probably longer when nurses deliver care and the frequency of attended return visits is probably slightly higher for nurses, compared to doctors. Other utilisation outcomes are probably the same. The effects of nurse‐led care on process of care and the costs of care are uncertain, and we also cannot ascertain what level of nursing education leads to the best outcomes when nurses are substituted for doctors.

Plain language summary

Nurses as substitutes for doctors in primary care

What is the aim of this review?

The aim of this Cochrane Review was to find out what happens when primary healthcare services are delivered by nurses instead of doctors. We collected and analysed all relevant studies to answer this question and found 18 studies for inclusion in the review.

What are the key messages of this review?

Delivery of primary healthcare services by nurses instead of doctors probably leads to similar or better patient health and higher patient satisfaction. Nurses probably also have longer consultations with patients. Using nurses instead of doctors makes little or no difference in the numbers of prescriptions and tests ordered. However, the impacts on the amount of information offered to patients, on the extent to which guidelines are followed and on healthcare costs are uncertain.

What was studied in this review?

In most countries, the population is growing older and more people have chronic disease. This means that the services that primary healthcare workers need to deliver are changing. At the same time, many countries lack doctors and other healthcare workers, or people struggle to pay for healthcare services. By using nurses instead of doctors, countries hope to deliver care of the same quality for less money.

In this review, we searched for studies that compared nurses to doctors for delivery of primary care services. We looked at whether this made any difference in patients’ health, satisfaction, and use of services. We also looked at whether this made any difference in how services were delivered and in how much they cost.

What are the main results of this review?

We included in this review 18 studies, mainly from high‐income countries. In some studies, nurses were responsible for all patients who came to the clinic or for all patients who needed urgent consultation. In some studies, nurses were responsible for patients with particular chronic diseases, or were responsible for providing healthcare education or preventive services to certain groups of patients. Included studies compared these nurses to doctors carrying out the same tasks.

Our review shows that nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths. In addition, patients probably have similar or better results in areas of health such as heart disease, diabetes, rheumatism, and high blood pressure. Patients also are probably slightly more satisfied with their care and may have a slightly better quality of life when treated by nurses.

This review also shows that, compared to doctors, nurses probably have longer consultations, and their patients are slightly more likely to keep follow‐up appointments. Studies found little or no difference in the number of prescriptions and there may be little or no difference in the numbers of tests and investigations ordered, or in patients’ use of other services. The effects of nurse‐led primary care on the amount of advice and information given to patients, and on whether guidelines are followed, are uncertain as the certainty of these findings is very low.

Our review suggests that the impacts on the costs of care of using nurses instead of doctors to deliver primary care are uncertain. We assessed the certainty of this finding as very low.

How up‐to‐date is this review?

We searched for studies that had been published up to March 2017.

Summary of findings

Background

Description of the condition

A range of existing and anticipated issues, including ageing populations, increased prevalence of chronic conditions and multi‐morbidity, increased emphasis on healthy lifestyle and prevention, and movement of healthcare services from hospitals to communities, have encouraged countries worldwide to develop new models of primary care delivery (Freund 2015; Roland 2014). As many tasks do not necessarily require the knowledge and skills of a doctor, using nurses to expand the capacity of the primary care workforce is a topic of increasing interest (Kooienga 2015; Maier 2016b). Substitution of nurses for doctors is one strategy for improving access to and efficiency and quality of care (NHS 2016; Perloff 2016), and advanced nursing practice roles are common in high‐, middle‐ and low‐income countries (Kooienga 2015; Maier 2016b). However, variation in primary care practice compositions is strong, and the same cadre might have different roles and authority in relation to practice depending on legislation, the healthcare system, and local practices (Freund 2015; Groenewegen 2015; Laurant 2009). Furthermore, reforms currently being implemented in many countries regarding nurses’ regulatory barriers or expansion of nurses’ scope of practice (e.g. in relation to prescribing medicines) suggest a shift in the boundaries between medicine and nursing (Maier 2016b).

Description of the intervention

Nurses in primary care may undertake many tasks traditionally performed by doctors. Tasks can be supplementary to those performed by doctors or can be substituted for doctors' tasks. The current review focusses on tasks in which nurses substitute for doctors, meaning that they provide the same services as doctors (Laurant 2009; Rashidian 2013), and is limited to care delivery for patients presenting with a physical complaint. These tasks may include diagnostics, treatment, referral to other services, health promotion, management of chronic diseases, or management of acute problems needing same‐day consultations. Contact with patients may take place in a primary health facility or in the home of the patient. Because people's understanding of what constitutes a nurse, as well as the educational levels of nurses, differs across countries (Kooienga 2015; Maier 2016b), we have included in this review all registered nurses who provide care as substitutes for doctors. When available, we have provided information on the educational levels of nurses in the included studies, based on the European Qualification Framework (EQF 2016). Moreover, the review aimed to include studies from high‐, middle‐, and low‐income countries, and we have described the impact of this approach on heterogeneity.

How the intervention might work

The expectation is that nurses substituted for doctors can do the following (e.g. Freund 2015; Kooienga 2015; Newhouse 2011; Rashidian 2013).

• Enhance the quality of services provided in primary care.

• Increase access to primary care services, as capacity increases.

• Reduce doctors' workload and thus free up time for doctors to take up more complex tasks.

• Reduce costs of care through lower salary costs for nurses and limited educational training, which is provided more quickly and is less expensive.

This last point was not, however, confirmed by a previous systematic review on this intervention (Martínez‐González 2015c). Gains in service efficiency may be achieved if doctors no longer provide the services they have delegated to nurses. This enables doctors to focus on complexity in their caseload and on utilising their advanced training and experience (Contandriopoulos 2015; Richardson 1999).

Why it is important to do this review

Advanced nursing practice roles have been developed worldwide, including in low‐, middle‐, and high‐income countries (Freund 2015; Kooienga 2015; Maier 2016b). The first advanced nursing roles were developed in the USA and Canada in the late 1960s/70s, in the UK in the 1980s, and in other high‐income countries in the 1990s and onwards (Laurant 2009). From the outset, nurses have been utilised to deliver primary care, traditionally in underserved areas and to vulnerable populations. Nowadays, their role has been extended to include other types of services in primary care (Poghosyan 2012), and this change has been implemented in a range of countries around the globe (Freund 2015; Kooienga 2015; Maier 2016b).

Nurses in advanced roles represent a substantial source of human capital to increase quality of care, access to (primary) care, and, as it is sometimes argued, efficiency of care, although recent reviews have not confirmed improvements in efficiency (Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c). It is believed that inclusion of nurses in advanced roles can ensure that the demand for healthcare services to address patient needs is properly met. Both practitioners and policy makers believe that to meet the challenges faced by primary care, a more robust healthcare workforce, including both doctors and nurses in advanced nursing roles, is needed (NHS 2016).

Although interest in expanding nursing roles and employing nurses as substitutes for doctors is increasing globally, underlying reasons for these initiatives differ depending on local context and circumstances (Savrin 2009). For example, the Health Resources and Services Administration in the USA, anticipating a shortage of doctors, has increased the amount of money available not only to train doctors but also to prepare nurse practitioners and physician assistants to support the primary care workforce (Petterson 2012). In addition, organisations such as the World Health Organization (WHO) have made several recommendations regarding ways to expand the role of nurses (WHO 2012). To enable policy makers to make informed decisions about healthcare delivery models, we need rigorous evidence on the quality of care, as well as on access and costs, associated with care provided by nurses compared with care provided by doctors.

Since this review was first published in 2005 (Laurant 2005), a large number of comparative studies have produced a stronger evidence base with regard to the effectiveness of nurse‐doctor substitution. Many new studies on nurses in primary care show increasing international interest in task shifting and in shifting of boundaries between medicine and nursing. Moreover, regulatory and educational reforms internationally support the trend towards advanced nursing roles in healthcare delivery and task shifting. The growth rate of the nursing workforce is now three times that of the workforce for doctors (nine times that for nurse practitioners), which provides an important opportunity to meet increasing demand within primary care (Maier 2016b).

This updated review adds value to recently published systematic reviews on this topic by excluding studies that do not focus solely on substitution, resulting in more accurate findings regarding the effectiveness of nurse‐doctor substitution specifically; and by using rigorous Cochrane methods. Additional insights provided by this update are important because results reported by some other reviews have been inconclusive (e.g. on costs), and because the ways in which primary healthcare services are organised have changed since our original review was published.

Objectives

Our aim was to investigate the impact of nurses working as substitutes for primary care doctors on:

• patient outcomes;

• processes of care; and

• utilisation, including volume and costs.

Methods

Criteria for considering studies for this review

Types of studies

Randomised trials (i.e. trials with random allocation of participants to intervention and control groups). We included controlled before‐after studies and non‐randomised trials in our previous Cochrane review (Laurant 2005). The number of available randomised trials has increased since that time; therefore, we decided to exclude controlled before‐after studies (n = 3) and non‐randomised trials (n = 3) from this update. Randomised trials provide more robust evidence on effectiveness, and including other study designs is unlikely to be worthwhile in light of the many randomised trials now available.

Types of participants

• Doctors: any kinds of doctors working in a primary care setting, including general practitioners, family doctors, paediatricians, general internists, and geriatricians. In this review, we use 'doctor' as the generic term for this cadre.

• Nurses: any qualified registered nurses working as substitutes for doctors in primary care. The definition of a qualified nurse is “a graduate who has been legally authorised (registered) to practice after examination by a state board of nurse examiners or similar regulatory authority” (WHO 2012). Included are nurse practitioners, clinical nurse specialists, advanced practice nurses, practice nurses, health visitors, etc. As the job title, education, and experience of nurses vary considerably among and within countries, we did not select nurses by virtue of their job title. We excluded only mental health nurses as this kind of substitution is addressed by the EPOC review on non‐specialist health workers for mental health (Ginneken 2013). We also excluded trainee nurses as they do not work to their full potential as a consequence of their traineeship.

• Patients: any persons presenting in primary care with a physical complaint.

This review is limited to primary healthcare services that provide first contact and ongoing care for patients with all types of physical health problems. It includes family practice, general practice, out‐patient care, and ambulatory primary care settings but excludes accident and emergency departments in hospitals. Patients presenting to accident and emergency departments in hospitals are not considered to be comparable to patients presenting for primary care services. These hospital departments generally deal with genuine life‐threatening emergencies and therefore are not considered an alternative to an appointment with a doctor in primary care.

Types of interventions

This review focusses on nurses working as substitutes for primary care doctors. Substitution refers to the situation wherein task(s) formerly performed by one type of professional (i.e. a doctor) are transferred to a different type of professional (i.e. a nurse), usually with the intention of reducing cost or addressing workforce shortages (Freund 2015; Laurant 2009). Substitution studies typically examine cases in which a nurse is responsible for providing the same health care as a doctor and compare the performance of these two practitioners. For example, study authors may compare a nurse‐led clinic for a particular disease or condition versus a doctor‐led clinic for that same disease or condition.

We excluded studies which evaluated nurses supplementing the work of primary care doctors. Supplementation refers to the situation wherein a nurse supplements or extends the care provided by a doctor by providing a new primary care service. Generally, the aim is to improve the quality of care rather than reduce cost or address workforce shortages. Supplementation studies typically compare usual care provided by a doctor versus an innovative service provided by a nurse working alongside a doctor. For example, researchers may compare a family practice with a nurse‐led diabetes clinic versus a family practice without such a clinic. This type of study risks confounding two aspects of care provision: type of service (specialised clinic vs routine consultation), and who provides that service (doctor or nurse).

Types of outcome measures

We considered three types of outcomes for inclusion in this review: patient outcomes; process of care outcomes; and utilisation outcomes, including both volume and costs.

Primary outcomes

Patient outcomes

• Mortality

• Health status (clinical outcomes and self‐reported outcomes)

• Satisfaction

• Quality of life

• Other (compliance, knowledge, preference for doctor or nurse)

Secondary outcomes

Process of care outcomes

• Practitioner adherence to clinical guidelines

• Practitioner healthcare activity (examinations, provision of advice)

Utilisation outcomes

Volume

• Frequency and length of consultations

• Number of return visits

• Number of prescriptions

• Numbers of tests and investigations

• Number of referrals to or frequency of use of other services

Costs

• Direct health service costs related to volume

• Indirect (societal) costs

Search methods for identification of studies

Electronic searches

We searched the following databases.

• Cochrane Central Register of Controlled Trials (CENTRAL; 2014, Issue 12), part of the Cochrane Library (www.cochranelibrary.com (searched 20.01.2015).

• MEDLINE In‐Process & Other Non‐Indexed Citations, MEDLINE Daily, MEDLINE and Ovid OLDMEDLINE 1946 to present, Ovid (searched 20.01.2015).

• Cumulative Index to Nursing and Allied Health Literature (CINAHL) 1981 to present, EbscoHost (searched 20.01.2015).

We performed an updated search in CENTRAL, MEDLINE, and CINAHL in March 2017. We have added one study to ‘Studies awaiting classification’ and will incorporate this study into the review at the next update.

Searching other resources

Grey literature databases

• The Grey Literature Report (http://www.nyam.org/library/online‐resources/grey‐literature‐report/) (searched 21.02.2017)

• OpenGrey (http://www.opengrey.eu/) (searched 21.02.2017)

Trial registries

• International Clinical Trials Registry Platform (ICTRP), World Health Organization (WHO) (http://www.who.int/ictrp/en/) (searched 21.02.2017)

• ClinicalTrials.gov, US National Institutes of Health (NIH) (http://clinicaltrials.gov/) (searched 21.02.2017)

We also searched the Science Citation Index and the Social Sciences Citation Index 1975 to present, for articles citing relevant studies, as well as Web of Knowledge (Thomson Reuters) (searched 27.01.2015) and the reference lists of all included papers and identified relevant reviews.

Please see Appendix 1 for strategies used and the PRISMA flow chart (Figure 1) for records retrieved, excluded, and included.

Data collection and analysis

Selection of studies

At least two review authors (among ML, NW, KW, EK, and AVV) independently screened search results at three levels: titles; abstracts to assess which studies potentially satisfied the inclusion criteria; and full‐text copies of papers that were potentially relevant. If we could not assess the paper for eligibility based on title or abstract, we obtained the full text. Where data was published in duplicate, we included these data only once in the review.

Data extraction and management

For this review, we designed a data extraction form that was based on the previously used standard form of the Cochrane Effective Practice and Organisation of Care Group (EPOC). At least two review authors (of ML, MB, NW, KW, EK, and AVV) independently abstracted data from each study and resolved differences by discussion.

If a single publication reported two or more separate studies, we extracted each study separately. If findings of a single study were spread across two or more publications, we extracted data from these publications as one. We extracted outcomes measured at different time points and presented in different publications, for example, at six months and two years after the intervention. We used the longest follow‐up in meta‐analyses. For each study with more than one control or comparison group for the nurse intervention, we reported only results for the control condition in which doctors provided the same intervention as the nurse.

Assessment of risk of bias in included studies

At least two review authors (among ML, MB, NW, KW, EK, and AVV) independently assessed risk of bias of each included study using the criteria suggested by EPOC (EPOC 2017). We assessed randomised trials for generation of allocation sequence, concealment of allocation, similar baseline outcome measurements, similar baseline characteristics, incomplete outcome data, blinding of participants, blinding of outcome assessors, protection against contamination, selective outcome reporting, and bias due to lack of power. We scored each study for risk of bias as follows: ’low’ if all key domains were scored as ’low risk’; ’unclear’ if one or two key domains were scored as ’unclear risk’; and ’high’ if more than two key domains were scored ’unclear risk’ or ’high risk’. When no information was available, we scored 'unclear risk'. For similar baseline characteristics and outcome measurements, we scored 'low risk' when baseline values were equal, or when analysis included a correction for differences in baseline values. We scored incomplete outcome data as low risk when follow‐up was ≥ 80% or when follow‐up was < 80%, with equal results attained by intention‐to‐treat (ITT) and per‐protocol (PP) analyses. With respect to blinding, we used the following approach. When investigators reported no blinding of patients and personnel, we scored 'unclear risk', because we do not know whether lack of blinding influenced study results. For some objective outcomes (e.g. mortality), blinding does not influence risk of bias, but for other outcomes in the same study (e.g. satisfaction), non‐blinding may influence outcomes.

We did not split the different outcomes for assessment of risk of bias within a study because the judgement of risk of bias was generally equal for all outcomes within a study. If the risk of bias judgement for a particular outcome was divergent, we commented on that.

We have shown assessments of risk of bias for included studies in the Characteristics of included studies table and have summarised this information in Figure 1 and Figure 2. We did not use risk of bias assessments in deciding which studies should be included in the meta‐analyses. However, we conducted sensitivity analyses by excluding studies with high risk of bias (see Sensitivity analysis). Furthermore, we used these assessments in interpreting study results and, particularly, in assessing the certainty of evidence for nurse‐doctor substitution.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Measures of treatment effect

The measure of effect size for continuous outcomes (patient satisfaction, disease activity score, pain, and length of consultation) was the standardised mean difference (SMD). The SMD is more appropriate than the mean difference (MD) in situations where the measurement instrument (i.e. the patient satisfaction scale) differs between studies. (See the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.) We used MDs for patient outcomes including blood pressure, cholesterol, and glycated haemoglobin (HbA1c).

The measure of effect size used for dichotomous outcomes was the risk ratio (RR). (See the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.) We converted odds ratios (ORs) to RRs using the built‐in calculator in RevMan 5.3 (RevMan 2014).

Unit of analysis issues

We included in the meta‐analyses three cluster‐randomised trials along with fifteen individually randomised trials. We included one cluster‐randomised trial that accounted for clustering in their analyses (Campbell 2014). Two of the included cluster‐randomised trials did not correct adequately for clustering (Chambers 1978; Spitzer 1973). We explored the impacts of these trials by performing a Sensitivity analysis.

Dealing with missing data

For missing or unclear information, we contacted study investigators to request clarification or additional information. For studies that reported continuous data but did not report standard deviations, we calculated these values from other available data such as standard errors, or imputed them using the methods suggested in Higgins 2011.

We extracted data from the ITT analysis when possible. If ITT data were not present, we excluded the study from meta‐analyses by performing a Sensitivity analysis.

Assessment of heterogeneity

Clinical settings, country contexts, and methodological diversity

We first made a qualitative assessment of the extent to which studies assessing a particular comparison were similar to one another. This included assessment of clinical settings, country contexts, and types of measurement scales to determine whether meta‐analysis was appropriate.

Statistical heterogeneity

We obtained an initial visual overview of statistical heterogeneity by scrutinising forest plots, looking at the overlap between confidence intervals around the estimate for each included study. In addition, we used the I2 statistic and confidence intervals to estimate and quantify heterogeneity.

Assessment of reporting biases

To reduce possible publication bias, we employed strategies to search for and identify relevant unpublished studies for inclusion. These strategies included searching the grey literature and prospective trial registration databases to overcome time‐lag bias.

We used funnel plots for outcomes reported by more than four studies to visualise whether data showed asymmetry. No plots showed asymmetry. However, we identified too few studies for reliable assessment of funnel plot asymmetry ‐ visually or quantitatively.

Data synthesis

To summarise the effectiveness of the nurse‐doctor substitution, we performed several meta‐analyses. We conducted statistical meta‐analyses using the RevMan 5.3 software distributed by Cochrane (RevMan 2014). For studies in which quantitative data were absent or were insufficient for calculation, we reproduced the data as presented in the additional tables, undertook a structured synthesis, and reported the findings narratively. We performed a meta‐analysis if the nature of the outcome and other key aspects of studies were similar. We used adjusted RRs if available in the article. When not available, we calculated RRs from events. For categorical outcomes, we calculated log RRs and standard errors (SEs) of log RRs for both individual and cluster‐randomised trials. We analysed together log RRs for individual randomised trials and adjusted log RRs for cluster‐randomised trials. We preferred RRs over ORs because interpretation is intuitive. When no cluster‐randomised trials were included in the meta analysis, we used RRs instead of log RRs.

We used a random‐effects meta‐analysis, which is known to be more conservative and more suitable in the presence of any heterogeneity (Kontopantelis 2012). Although we expected substantial heterogeneity in some cases, which could be attributed to differences among populations, interventions, comparators, outcomes, and settings, we are aware that detection of existing heterogeneity can be problematic for meta‐analysis that includes a small number of studies (Kontopantelis 2013). Therefore, for greater transparency, we reported 95% confidence intervals of the I2 statistic, obtained under an inverse variance DerSimonian‐Laird random‐effects model for continuous outcomes, and a Mantel‐Haenszel/DerSimonian‐Laird random‐effects model hybrid for dichotomous outcomes.

'Summary of findings'

We used the GRADE approach to assess the certainty of evidence related to each of the key outcomes (Schünemann 2009). We used the GRADE profiler to import data from Review Manager 5.3 and create Table 1 (RevMan 2014; GRADEpro GDT 2015).

for the main comparison

For assessments of the overall certainty of evidence for each outcome, we downgraded the evidence from 'high certainty' by one level for serious (or by two levels for very serious) study limitations (risk of bias), indirectness of evidence, serious inconsistency, imprecision of effect estimates, or potential publication bias. We provided justification for decisions to downgrade or upgrade ratings by inserting footnotes into the table and made comments to aid readers' understanding of the review when necessary. We used plain language statements to report these findings in the review.

We used these assessments, along with evidence for absolute benefit or harm of the interventions and the sum of available data on all critical and important outcomes from each study included for each comparison, to draw conclusions about the effectiveness of nurse‐led primary care. Table 1 consists of critically important clinical and functional outcomes identified in the selected trials.

When judging the importance of SMDs, we acknowledged that 0.2 represents a slight effect, 0.5 a moderate effect, and 0.8 a significant effect (Guyatt 2008; Higgins 2011).

Subgroup analysis and investigation of heterogeneity

During the review process, we identified several factors that might explain heterogeneity in review findings, including type of nurse (i.e. nurse, registered nurse, nurse practitioner, specialised nurse); characteristics of the intervention and the comparator (i.e. total substitution, partial substitution); study size; duration of follow‐up; type of care (i.e. single contact, series, urgent care); range of patient complaints (i.e. all patients or particular patient groups); and setting. We undertook these as exploratory, hypothesis‐generating analyses because these factors were not identified a priori and several potentially explanatory factors were considered. We considered undertaking a subgroup analysis based on nurse title as described in the included studies, as has been done in other systematic reviews (Martínez‐González 2014a). However, we have little information about exact role definitions and educational levels of nurses in the different trials, and we know that job titles differ among countries; therefore, we decided it was not possible to create clear and valid subgroups for subgroup analyses.

Results

Description of studies

Results of the search

We identified a total of 4831 articles from electronic and supplementary searches. We excluded 4741 articles following a review of titles and abstracts and retrieved and assessed the full text of 90 articles. We excluded 78 full‐text articles that investigated the role of nurses working as supplements to primary care doctors and excluded two additional studies that involved a mix of primary and hospital care. Nine randomised trials met the inclusion criteria, and we included them in this update. We performed an updated search in CENTRAL, MEDLINE, and CINAHL in March 2017. We have added one study to ‘Studies awaiting classification’ and will incorporate this study into the review at the next update. We have presented the study flow diagram in Figure 1.

Risk of bias in included studies

We prepared an assessment of risk of bias for each trial and illustrated final judgements for the ten criteria in Figure 2 and Figure 3. All studies had some methodological shortcomings, in most instances related to unclear risk of bias for different criteria. We judged only one study to be at high risk of bias for more than one criterion (Mundinger 2000). The criteria most commonly assessed as having unclear risk of bias were blinding of personnel, outcome assessment, and selective reporting. The criterion most commonly assessed as having high risk of bias was contamination (Lewis 1967; Mundinger 2000; Spitzer 1973; Voogdt‐Pruis 2010).

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Most studies stated that participants or practices (in case of cluster randomisation (Campbell 2014)) were assigned randomly, according to a computerised randomisation scheme. Twelve of the included studies met the ‘low risk of bias’ criteria for random sequence generation. Most of these studies used a computer (Chan 2009; Hemani 1999; Voogdt‐Pruis 2010), and some used envelopes for this purpose (Dierick‐van Daele 2009; Houweling 2011; Larsson 2014; Shum 2000). For six studies, the risk of bias for random sequence generation was unclear owing to poor reporting. Campbell 2014 used a random component in the sequence process, and 10 of 15 practices withdrew after randomisation, which made the risk of bias unclear.

Baseline values

Most studies provided similar outcome measurements between the two study arms at baseline or corrected for differences in baseline values. Hemani 1999 did not assess baseline characteristics, and study arms differed for one or two outcome measures in Larsson 2014.

Blinding

Risk of performance bias was low in two studies (Hemani 1999; Voogdt‐Pruis 2010). In Hemani 1999, personnel did not know which patients were included in the study, and Voogdt‐Pruis 2010 collected data retrospectively and asked patients for their consent after one year. For all other studies, we judged the risk of performance bias as unclear because no information was available. We expect that patients and personnel were not blinded in these studies because the care provider constitutes the intervention. Whether this lack of blinding influences outcomes is unclear. Three studies satisfied the criteria for blinding of outcome assessors (Chan 2009; Iglesias 2013; Ndosi 2013). These studies provided independent researchers who where blind to group assignment when measuring outcomes. Most studies did not provide sufficient information on blinding of outcome assessment; we therefore assessed them as having unclear risk of detection bias.

Incomplete outcome data

Three studies reported follow‐up less than 80% (Chambers 1978; Mundinger 2000; Venning 2000); we therefore judged these studies to have high risk of bias for incomplete outcome data. In most studies, 80% or more of the initial participants completed the study. Risk of bias due to incomplete outcome data was unclear in Hemani 1999 because of limited reporting about follow‐up. Ndosi 2013 reported follow‐up of less than 80%. However, investigators performed both ITT and PP analyses and reported the same results (Ndosi 2013).

Selective reporting

We judged two studies to have low risk for selective outcome reporting bias (Campbell 2014; Ndosi 2013). A protocol was available for each study, and these papers reported predefined outcome measures. Absence of study protocols to confirm reporting of all intended outcomes led to the unclear judgement in all other studies.

Effects of interventions

See: Table 1

Patient outcomes

A total of 18 trials investigated patient outcomes (Campbell 2014; Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Ndosi 2013; Sanne 2010; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010) (Table 2).

1

Patient outcome: health status

We have grouped patient outcomes into the following categories: mortality, health status outcomes, satisfaction and preferences, quality of life, and other patient outcomes.

Mortality

Eight trials evaluated mortality (Campbell 2014; Hemani 1999; Lattimer 1998; Ndosi 2013; Sanne 2010; Shum 2000; Spitzer 1973; Voogdt‐Pruis 2010). Meta‐analysis of data from these trials suggests that nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. Among those people who received doctor‐led care, 6 per 1000 people died. Among those people who received nurse‐led care, between 4 and 6 people per 1000 died (RR 0.77, 95% CI 0.57 to 1.03, low certainty evidence). Data show no evidence of statistical heterogeneity (I2 = 0%, 95% CI 0 to 68; Analysis 1.1). The evidence is of low certainty owing to a wide confidence interval that includes no effect (imprecision) and clinical heterogeneity, as the trials contributing to this estimate are quite varied (some focus on people with specific health issues and others on more generalist primary care attenders). Excluding from the meta‐analysis a trial assessed as cluster‐randomised did not greatly change the result (RR 0.56, 95% CI 0.33 to 0.95) (Lattimer 1998). Results did not differ considerably in the other sensitivity analyses.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 1 Mortality.

Other health status outcomes

We grouped health status outcomes into clinical outcomes (e.g. blood pressure, cholesterol, glycated haemoglobin (HbA1c)) and self‐reported measurements of health status, including measures related to physical functioning (e.g. pain, Disease Activity Score (DAS)) and lifestyle factors (e.g. smoking, alcohol consumption, exercise).

Clinical outcomes

Three trials focussing on patients with cardiovascular disease or diabetes evaluated clinical outcomes (Houweling 2011; Mundinger 2000; Voogdt‐Pruis 2010). Meta‐analyses for blood pressure levels suggest that, compared to doctor‐led care, nurse‐led primary care probably slightly improves blood pressure outcomes for both systolic blood pressure (MD ‐3.73, 95% CI ‐6.02 to ‐1.44, moderate‐certainty evidence; Analysis 1.5) and diastolic blood pressure (MD ‐2.54, 95% CI ‐4.57 to ‐0.52, moderate‐certainty evidence; Analysis 1.6). For both outcomes, data show no evidence of statistical heterogeneity (systolic blood pressure: I2 = 0%, 95% CI 0 to 90; diastolic blood pressure: I2 = 0%). Assessment of moderate‐certainty evidence is due to high risk of bias in one of the included studies (Mundinger 2000). Results did not change considerably under Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 5 Systolic blood pressure.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 6 Diastolic blood pressure.

A meta‐analysis for HbA1c suggest that nurse‐led primary care probably leads to similar outcomes as doctor‐led care and a meta‐analysis for cholesterol suggest that nurse‐led primary care leads to similar outcomes as doctor‐led care for patients with heart failure or diabetes (HbA1c levels: MD 0.08, 95% CI ‐0.25 to 0.41, moderate‐certainty evidence; Analysis 1.8; total cholesterol: MD ‐0.15, 95% CI ‐0.32 to 0.02, high‐certainty evidence; Analysis 1.7). For both outcomes, data show no evidence of statistical heterogeneity (cholesterol: I2 = 0%, 95% CI 0 to 90; HbA1c: I2 = 0%). The assessment of moderate‐certainty evidence for HbA1c evidence is due to high risk of bias in one of the included studies (Mundinger 2000). Results did not change considerably under Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 7 Total cholesterol.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 8 HbA1c.

Self‐reported measurements of health status

Twelve trials provided self‐reported measurements of health status (Chambers 1978; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Larsson 2014; Lewis 1967; Moher 2001; Ndosi 2013; Sanne 2010; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). Two trials among patients with rheumatological diseases (Larsson 2014; Ndosi 2013) assessed the outcomes disease activity in rheumatoid arthritis and pain. Meta‐analyses for DAS and pain suggest that nurse‐led primary care, compared to doctor‐led care, for patients with rheumatological disease probably leads to similar outcomes for DAS and pain (DAS: MD 0.04, 95% CI ‐0.17 to 0.24, moderate‐certainty evidence; Analysis 1.9; pain: MD 0.76, 95% CI ‐3.85 to 5.38, moderate‐certainty evidence; Analysis 1.3). For both outcomes, there was no evidence of statistical heterogeneity (DAS: I2 = 1%; pain: I2 = 0%). The evidence is of moderate certainty owing to indirectness, as only patients with rheumatoid arthritis were included. Results did not change considerably under Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 3 Pain.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 9 Disease Activity Score.

We included in a meta‐analysis three studies assessing physical functioning. Results suggest that, compared to doctor‐led care, nurse‐led primary care may lead to little or no difference in physical functioning (RR 1.03, 95% CI 0.98 to 1.09, low‐certainty evidence; Analysis 1.2). Results showed statistical heterogeneity (I2 = 62%, 95% CI 0 to 87, P = 0.07). The evidence is of low certainty owing to inconsistency and high risk of bias. Results did not change considerably when a trial assessed as having high risk of bias (Chambers 1978) was excluded under Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 2 Physical function (better vs not better).

In addition, studies measured a large number of other outcomes related to health status and lifestyle. It was not possible to pool these results because of the wide range of outcomes assessed, but results suggest that care provided by nurses was at least as good as care provided by doctors. We have summarised the details in Table 2.

Satisfaction and preferences

Ten trials measured satisfaction with care (Campbell 2014; Dierick‐van Daele 2009; Iglesias 2013; Larsson 2014; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000). This outcome was assessed in many different ways across trials; therefore we could include only seven trials in a meta‐analysis (Campbell 2014; Dierick‐van Daele 2009; Iglesias 2013; Larsson 2014; Mundinger 2000; Shum 2000; Venning 2000). This showed that patient satisfaction is probably slightly higher in nurse‐led primary care than in doctor‐led primary care (SMD 0.08, 95% CI 0.01 to 0.15, moderate‐certainty evidence; Analysis 1.10). The evidence is of moderate certainty owing to inconsistency (I2 = 56%, 95% CI 23 to 74), suggesting that the extent to which nurse‐led care increased patient satisfaction varied considerably with the context of care. Results did not change considerably under Sensitivity analysis. Findings of trials not included in this meta‐analysis also suggest that patients are probably at least as satisfied with nurse‐led care as with doctor‐led care. Table 3 summarises the data for all trials that assessed this outcome.

2

Patient outcome: satisfaction and preference

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 10 Patient satisfaction.

In addition, investigators measured a large number of other outcomes related to patient satisfaction and preferences. It was not possible to pool these results, but findings suggest that patients are at least as satisfied with nurses as with doctors. We have summarised details in Table 3.

Quality of life

Six trials evaluated quality of life (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Houweling 2011; Mundinger 2000; Ndosi 2013). Meta‐analysis of data from these trials suggests that quality of life may be slightly higher for people receiving nurse‐led primary care, compared to doctor‐led primary care (SMD 0.16, 95% CI 0.00 to 0.31, low‐certainty evidence; Analysis 1.4). The evidence is of low certainty owing to inconsistency (I2 = 85%, 95% CI 69 to 93) and to imprecision, as the confidence interval touches on the null. The heterogeneity was caused by one trial (Chan 2009), which included a specific patient group (i.e. people who had experienced dyspepsia after direct access gastroscopy). After we excluded this trial, we found that there may be little or no difference in quality of life among patients receiving nurse‐led primary care, compared to doctor‐led primary care (SMD 0.02, 95% CI ‐0.01 to 0.05). The results also did not show evidence of heterogeneity or change considerably under other Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 4 Quality of life.

Other patient outcomes

Investigators measured a large number of other patient outcomes, including patient knowledge (understanding the health issue) and patient enablement (coping with his or her health issues). It was not possible to pool these results, but findings suggest that care provided by nurses was probably at least as good as care provided by doctors. We have summarised the details in Table 4.

3

Patient outcome: compliance and other

Process of care outcomes

Ten trials investigated process of care outcomes (Campbell 2014; Dierick‐van Daele 2009; Houweling 2011; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). We have summarised the data in Table 5. Owing to the large variety of approaches used in measuring the process of care, we did not judge it appropriate to pool these data in a meta‐analysis. The individual trial results show some differences between nurses and primary care doctors in process of care measures. For example, investigators reported that nurses gave more advice/information to patients and adhered to guidelines more frequently. However, the quality of patient examinations appeared to be similar between nurses and doctors. Overall, we assessed this evidence to be of very low certainty as the results were non‐comparable and we could not calculate an overall effect size.

4

Process of care outcomes

Utilisation outcomes

Sixteen trials measured utilisation and costs (Campbell 2014; Chan 2009; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Moher 2001; Mundinger 2000; Ndosi 2013; Shum 2000; Spitzer 1973; Venning 2000; Voogdt‐Pruis 2010). The range of outcomes varied across trials and can be grouped into four categories: length and frequency of consultations; numbers of prescriptions, tests, and investigations ordered; use of other healthcare services, such as hospital admissions or referral to other professionals (Table 6); and costs (Table 7). Findings for each of these categories are presented below.

5

Utilisation outcomes

6

Utilisation; cost outcomes

Consultations

Seven trials investigated consultation length (Dierick‐van Daele 2009; Houweling 2011; Iglesias 2013; Lewis 1967; Ndosi 2013; Shum 2000; Venning 2000). Four trials provided sufficient data for a meta‐analysis on consultation length (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000). This analysis suggests that nurses probably have longer consultations than doctors (SMD 0.38, 95% CI 0.22 to 0.54, moderate‐certainty evidence; Analysis 1.11). The evidence is of moderate certainty owing to serious inconsistency (I2 = 90%, 95% CI 80 to 95). The extent of heterogeneity suggests that differences in consultation length varied considerably with the context of care. On average, consultations with nurses were 39% (95% CI 30% to 52%) longer than those with doctors. Results did not change considerably under Sensitivity analysis. Findings of the trials not included in the meta‐analysis also suggest that consultations in nurse‐led care were probably longer than those in doctor‐led care.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 11 Length of consultation.

Nine trials investigated consultation rates in primary care (including overall consultation rates, return visits for whatever reason, and home visits) (Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Venning 2000). Three trials provided sufficient data for a meta‐analysis of scheduled return visits (Dierick‐van Daele 2009; Shum 2000; Venning 2000), and four trials on attended return visits (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000). Data show that there may be little or no difference in scheduled return visits (RR 1.31, 95% CI 0.89 to 1.94, low‐certainty evidence; Analysis 1.12). The number of attended return visits is higher in nurse‐led primary care than in doctor‐led primary care (RR 1.19, 95% CI 1.07 to 1.33, high‐certainty evidence; Analysis 1.13). For scheduled return visits, the evidence is of low certainty owing to serious inconsistency (I2 = 86%, 95% CI 54 to 92) and imprecision (wide confidence interval). Results did not change considerably under Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 12 Scheduled return visits.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 13 Attended return visit.

Findings of the trials not included in the meta‐analysis were congruent with those reported above . Furthermore, the workload of doctors was probably slightly less where care was led by nurses, compared to where it was led by doctors, as were waiting times for patients in the waiting room. Table 6 summarises the data for all trials that assessed this outcome.

Numbers of prescriptions, tests, and investigations

Seven trials evaluated rates of prescriptions, tests, and investigations (Dierick‐van Daele 2009; Hemani 1999; Iglesias 2013; Moher 2001; Shum 2000; Venning 2000; Ndosi 2013). Four trials provided sufficient data for a meta‐analysis on the number of prescriptions given (Dierick‐van Daele 2009; Iglesias 2013; Shum 2000; Venning 2000), and four trials on the number of tests and investigations (Dierick‐van Daele 2009; Hemani 1999; Venning 2000; Ndosi 2013). Meta‐analyses of data from these trials suggest little or no difference between nurse‐led care and doctor‐led care in the number of prescriptions given (RR 0.99, 95% CI 0.95 to 1.03, high‐certainty evidence; Analysis 1.14).

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 14 Prescription ordered.

The findings also show that there may be little or no difference in the number of tests/investigations (RR 0.95, 95% CI 0.59 to 1.51, low‐certainty evidence; Analysis 1.15). The evidence is of low certainty owing to serious inconsistency (I2 = 76%, 95% CI 23 to 86) and a wide confidence interval, suggesting that the number of ordering tests/investigations varied between nurse‐led care and doctor‐led care according to the context of care.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 15 Investigations.

The findings of trials not included in the meta‐analyses also suggest little or no difference between nurse‐led and doctor‐led care in numbers of prescriptions and investigations/tests. Table 6 summarises the data for all trials that assessed this outcome.

Use of other services

Thirteen trials investigated people's use of services, including referrals, specialty visits, and hospital admissions (Campbell 2014; Dierick‐van Daele 2009; Hemani 1999; Houweling 2011; Iglesias 2013; Larsson 2014; Lattimer 1998; Lewis 1967; Mundinger 2000; Ndosi 2013; Shum 2000; Venning 2000; Voogdt‐Pruis 2010). Of these trials, four provided sufficient data for a meta‐analysis on hospital referral (Houweling 2011; Lattimer 1998; Mundinger 2000; Venning 2000), five for a meta‐analysis on attendance at accident and emergency units (Campbell 2014; Iglesias 2013; Lattimer 1998; Mundinger 2000; Shum 2000), and three for a meta‐analysis on hospital admission (Lattimer 1998; Mundinger 2000; Ndosi 2013). These meta‐analyses suggest that there may be little or no difference between nurse‐led care and doctor‐led care in the likelihood of hospital referrals (RR 0.90, 95% CI 0.54 to 1.49, low‐certainty evidence; Analysis 1.16), as well as little or no difference in attendance at accident and emergency units (RR 1.00, 95% CI 0.91 to 1.09, high‐certainty evidence; Analysis 1.17). In addition, there may be little or no difference in hospital admissions (RR 1.04, 95% CI 0.78 to 1.39, low‐certainty evidence; Analysis 1.18). For referrals, evidence is of low certainty owing to inconsistency (I2 = 50%, 95% CI 0 to 86) and a wide confidence interval, suggesting that the extent to which the frequency of referrals differs between nurse‐led care and doctor‐led care varied with the context of care. For hospital admissions, the evidence is of low certainty owing to risk of bias in one of the included trials (Mundinger 2000) and a wide confidence interval. Results did not change considerably in the Sensitivity analysis.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 16 Hospital referral.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 17 Attendance at accident and emergency.

Analysis

Comparison 1 Doctor‐nurse substitution study results, Outcome 18 Hospital admission.

Findings of the trials not included in the meta‐analyses also suggest little or no difference between nurse‐led and doctor‐led care on use of other services. Table 6 summarises the data for all trials that assessed this outcome.

Discussion

Summary of main results

This review identified 18 randomised trials evaluating the impact of nurses working as doctors’ substitutes. One study was from a middle‐income country, and all of the other studies were from high‐income countries. The type of nursing cadres involved in the studies was often unclear or varied between and even within studies.

Findings suggest that care delivered by nurses, compared to care delivered by doctors, probably leads to similar or better health outcomes for a broad range of patient conditions (moderate‐certainty evidence).

• Nurse‐led primary care may lead to slightly fewer deaths among certain groups of patients, compared to doctor‐led care. However, the results vary and it is possible that nurse‐led primary care makes little or no difference to the number of deaths.

• Blood pressure outcomes are probably slightly improved in nurse‐led primary care. Other clinical or health status outcomes are probably similar.

• Patient satisfaction is probably slightly higher in nurse‐led primary care. Quality of life may be slightly higher.

We are uncertain of the effects of nurse‐led care, compared to doctor‐led care, on processes of care such as patient education and adherence to guidelines. The effect of nurse‐led care on utilisation is mixed and depends on the type of outcome. Consultations are probably longer in nurse‐led primary care, and numbers of attended return visits are slightly higher for nurses than for doctors. We found little or no difference between nurses and doctors in the number of prescriptions and attendance at accident and emergency units. There may also be little or no difference in the number of tests and investigations, hospital referrals and hospital admissions between nurses and doctors. We are uncertain of the effects of nurse‐led care on the cost of care because the certainty of this evidence was assessed as very low.

An overview can be found in Table 1.

Overall completeness and applicability of evidence

Several issues need to be considered when one is making judgements about the applicability of these findings in primary care systems. First, we were able to identify a large number of studies published up to March 2017, which were sufficient to address all objectives of the review. These studies are highly varied in terms of types of nurses (with regard to both educational level and nurses' roles), healthcare systems, and geographical settings, and they examine care provided to general patient populations as well as to specific groups of patients, such as people with cardiovascular disease, diabetes and rheumatological diseases. Next, we found a large variation in outcome measures. For a number of outcomes there were only a few contributing studies whereas for some other outcomes a relative large number of studies contributed to the evidence. Furthermore, often details (such as nursing education level) were missing from study reports. Therefore we were not able to conduct planned subgroup analyses. As a result, it is not possible to draw conclusions on the influence of nurse type on outcomes. In addition, all but one of the included studies were conducted in high‐income country settings. Second, in some studies, interventions in nurse‐led and doctor‐led groups were somewhat different. For example, nurses had protocols or were offered a computerised decision tool, and doctors were not (Campbell 2014; Houweling 2011; Iglesias 2013; Lattimer 1998). In other studies, nurses’ patients were given an appointment but doctors’ patients were only advised to see their doctor (Chan 2009), or nurse‐led care included a longer time slot for consultations (Ndosi 2013). These differences in the interventions provided might have influenced study outcomes. Last, over the ten years since our previous review was published, primary care services have changed considerably in many settings. However, we did not identify a trend in types of nurse substitutions for doctors or in changes in outcomes assessed that might reflect changes in primary care services. The reasons for this are not clear.

Quality of the evidence

This review included studies from a wide range of nursing levels, patient groups, and countries. We were able to identify evidence on many different outcomes, but certainty of this evidence varies. All studies had some methodological shortcomings, such as contamination and lack of blinding both patients and personnel, which sometimes led to downgrading of the evidence owing to risk of bias. Although lack of blinding is considered a shortcoming, blinding is often not possible for organizational interventions, such as the substitution of one kind of health care provider with another. While the impact of this on outcomes is unclear, we believe the impact on patient satisfaction and experiences with care is likely to be limited. For example, a Dutch study which evaluated the impact of nurse practitioners in acute primary care settings showed that patients often do not know the profession of their care provider, even when the care provider had introduced themselves at the start of the consultation (only 18% of the patients treated by nurse practitioners remembered this) (Wijers 2013). The study suggests that patients do not judge the practitioner on the basis of their profession but rather on their competencies. Not blinding personnel may affect collaboration, as acceptance of a new professional is one of the main factors influencing skill mix changes (Laurant 2009). This influence could go either way: when a new professional is first introduced, other professionals and patients may be uncertain about the competencies of the new professional. However, over time the new professional may be accepted by both patients and other professionals. This has been shown in a recent study in which patients were more satisfied with the care provided by nurses over time (Wijers 2013). It is unclear whether this effect might also impact on patient outcomes, process of care outcomes and resource utilization.

For many meta‐analyses, measures of statistical heterogeneity were high indicating inconsistency across the included trials. Even where statistical heterogeneity was not detected, clinical heterogeneity may be present due to the range of types of nurses, health issues and settings included in the review (Kontopantelis 2013). Wide confidence intervals (imprecision) were another common reason to downgrade.

For some studies and outcomes, we were not able to conduct meta‐analyses owing to the diversity of the outcomes assessed. For these studies, we could only describe the results narratively, which made drawing overall conclusions difficult.

Potential biases in the review process

Our search strategy was designed to maximise sensitivity (detecting relevant research) at the expense of specificity (excluding irrelevant research). Even so, relevant research proved difficult to identify, and some studies may have been missed.

We conducted this review according to Cochrane standards. Therefore, we are confident in the quality of the review itself. Although publication bias cannot be ruled out in this area (Egger 1997), it seems unlikely that this bias could be substantial, as the clinical and research communities are equally interested in whether nurses perform as well as or better than doctors, or the reverse.

Agreements and disagreements with other studies or reviews

Results of this update are similar to those of the original review (Laurant 2005) in terms of health outcomes for patients, process of care, and resource utilisation.

Several other published reviews have examined nurses in primary care (Bonsall 2008; Hollinghurst 2006; Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c; Naylor 2010; Newhouse 2011; Swan 2015). Although the findings of our current review are generally consistent with those of other reviews, differences in findings might be explained by differences in review methods. Our review is most closely related to the reviews of Martínez‐González et al. (Martínez‐González 2014a; Martínez‐González 2014b; Martínez‐González 2015a; Martínez‐González 2015b; Martínez‐González 2015c). Although those review authors used inclusion criteria similar to ours, we noted differences in the included studies. There are several possible explanations for these differences. Firstly, there may be differences in the way the type of nurse role revision was labelled, and specifically whether this change was assessed as substitution or supplementation. Other systematic reviews included some studies that we assessed as involving nurses supplementing care provided by doctors rather than taking over tasks of doctors through substitution (Andryukhin 2010; Denver 2003; Du Moulin 2007; Fairall 2012; Hesselink 2004; Hiss 2007; Jarman 2002; Kernick 2000; Kernick 2002; Kuethe 2011; Winter 1981). In these supplementation studies, the intervention nurses provided an intervention that was complementary to usual care or both nurses and physicians were both involved in patient care, providing care as a team rather than providing care separately . Secondly, we included several studies that were not included in the reviews of the Martínez‐González team (Chambers 1978; Lattimer 1998; Moher 2001; Sanne 2010; Spitzer 1973). Only one of these five studies (Spitzer 1973) was listed in the table presenting the reasons for exclusions. Martinez‐Gonzalez team argued this study was not real substitution due to the contamination that occurred during the trial, with 30% of patients also treated by physicians at the end of the study. We judged contamination as high risk in this study, but not as a reason to exclude the study. The other four studies (Chambers 1978; Lattimer 1998; Moher 2001; Sanne 2010) were not included in the full text screening of papers by Martinez‐Gonsalez. It is therefore unclear whether these papers were not identified due to differences in search strategies or whether these papers were excluded at the title and abstract screening stage of the Martinez‐Gonsalez reviews. We have presented in Table 8 the key methodological differences between our review and the other reviews mentioned here.

7

Methodological differences with published reviews on care delivered by nurses compared to doctors in primary care

Several reviews found similar results to ours in terms of reductions in mortality in nurse‐led primary care compared to doctor‐led primary care, in particular in ongoing care and non‐urgent care provided by nurse practitioners (Martínez‐González 2014a; Swan 2015). However, one review (Newhouse 2011) showed that mortality rates were similar across these cadres, possibly owing to differences in review inclusion criteria. All other reviews described results similar to ours in terms of equal or higher health status for patients who received care from nurses compared to doctors (Martínez‐González 2015a; Newhouse 2011; Swan 2015).

Other reviews also found that nurse‐led care probably leads to higher patient satisfaction (Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2014a; Swan 2015 ); slightly higher quality of life (Martínez‐González 2014a); longer consultation length; and higher rate of return visits (Hollinghurst 2006; Horrocks 2002; Martin‐Misener 2015; Martínez‐González 2015b), compared to doctor‐led care. Our finding that there is little or no difference between nurses and doctors in frequency of prescriptions, tests, and investigations, and in patients’ use of other services is similar to that of the Martínez‐González 2015b review. However, Horrocks 2002 found that nurses‐led care was associated with more investigations but an equal number of prescriptions. It is likely, though, that the findings of the Horrocks 2002 review are now out of date.

While we were uncertain of the effects of nurse‐led care on the cost of care most other reviews (Hollinghurst 2006; Martin‐Misener 2015; Martínez‐González 2015b; Swan 2015)reported that there may be little or no difference in costs of care between nurse‐led care and doctor‐led care. Naylor 2010 and Newhouse 2011 indicated that nurse care was associated with lower costs. This difference might be explained by a focus on the USA only (Newhouse 2011), by the inclusion of non‐randomised trials, and by a focus on advanced nurses and nurse practitioners (Naylor 2010; Newhouse 2011). Authors of all reviews agree that evidence of effects of nurse‐led care on costs of care is of low quality.

Authors' conclusions

Implications for practice

Overall, nurse‐doctor substitution in primary care for provision of first contact care (including urgent care), ongoing care for all presenting physical complaints, and follow‐up of patients with a particular chronic condition has been relatively well evaluated. Nurse‐doctor substitution for preventive services and health education in primary care has been less well studied.

This review shows that trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide care that is equal to or of better quality than that provided by primary care doctors, and probably achieves equal or better health outcomes for patients. Nurses probably provide more health advice to patients, and probably achieve slightly higher levels of patient satisfaction, compared to primary care doctors (Table 1).

From this review, we cannot conclude whether it is better to deploy nurses providing care for a broad range of health issues or nurses who target groups of patients. Both approaches seem possible, with at least equal quality of care. Futhermore, the authors of this review cannot draw conclusions on the level of nursing education that leads to the best outcomes when nurses are substituted for doctors. In our review, the educational level of nurses was often unknown. In addition, studies often included a range of nurse roles and types, so we were not able to explore within our review whether evidence shows differences by type of nurse or by nursing role (Maier 2016b).

Whether nurse‐doctor substitution leads to substantial savings or whether nurse‐doctor substitution is cost‐effective remains unclear. Savings on nurse salaries may be offset by nurses' longer consultations and nurse rates as compared to doctor rates (Table 1). On the other hand, nurses probably adhere better to guideline recommendations, and their patients are probably more likely to attend return visits, which may positively affect health outcomes and reduce costs over the medium to long term.

Our review focussed on differences in outcomes between care provided by nurses and care provided by doctors. Although the included studies show effects of an independent practice role for nurses, it is likely that the quality of patient care overall is determined by overall functioning of the primary care team, including nurses, doctors, and other healthcare providers. Only three studies in our review assessed the impact of nurses on doctor behaviour. Policy makers should be aware that implementing nurse substitution in primary care teams may have an influence on the functioning and quality of care delivered by the entire care team.

Implications for research

Although this review includes a large number of studies, several important research questions remain.

The methodological quality of recent included studies is still variable. Future studies should seek to maximise the numbers of included healthcare providers, rather than the numbers of patients, to reduce the effect of any individual provider on outcomes. Moreover, studies with longer follow‐up periods are needed to provide better insights into impact on health status. For a full understanding of the impact of nurses in primary healthcare teams, we need deeper insights into the functioning of the entire team. Qualitative studies may be useful and could explore questions such as how nurses and doctors work as a team, how they interact, how their roles and responsibilities are defined, and how these agreements on roles and responsibilities affect nurse and doctor behaviours (Rashidian 2013). Further research is needed to enhance understanding of the limits of substitution, and to explore optimal models of collaboration and deployment of doctors and nurses as part of primary healthcare teams.

All studies except one were conducted in high‐income countries, and it is not clear whether results from this single study can be generalised to populations and health systems in middle‐ or low‐income countries. More research in middle‐ and low‐income countries is needed. Moreover, the influence of nursing education level on effects of nurse‐doctor substitution is poorly understood. Reasons include lack of international educational standardisation and insufficient reporting of nursing levels in research papers.

Since mortality is very important outcome, and the results of this review show important impacts, with mortality probably being decreased in nurse‐led primary care, this outcome should be assessed in future studies.

Costs, particularly societal costs, have not been well investigated, despite the widely held view that nurse‐led care will generate savings. Most studies have major limitations in cost evaluation. Future studies of nurse‐doctor substitution should give more attention to its financial aspects, for example, by performing cost‐effectiveness analyses. Related to this is the question of what impact changes in nurses' work have on the behaviours of doctors and on their workload. Only three of the included studies evaluated this, despite the widely held view that nurses can ‘save’ doctors’ time.

Authors of future reviews about nurse‐doctor substitution must take into account that healthcare services change extensively over time, and that new treatments and innovations may affect healthcare delivery. Organisational changes such as nurse‐doctor substitution are complex and should be treated in a way that leads to a well‐informed understanding of mechanisms and how these may impact outcomes (Salter and Kothari 2014).

Feedback

Comments submitted via Cochrane Library, 17 July 2018

Summary

Comment #1: I have never witnessed so many "probablies" in the conclusion of a cochrane review...These are probably conclusions. There is probably some bias. Cochrane, you need to do better.

Comment #2: It was probably written by nurses... I see an OBgyn.. not sure of their validity in this..

Reply

Response to Comment #1: We are aware of the many ‘probablies’ in the conclusion of our review. This is the result of GRADE assessments of the certainty of the evidence for the outcomes included in the review. For example, we had to downgrade the certainty of the evidence in some instances due to flaws in how the included studies were conducted (i.e., risk of bias). In line with Cochrane EPOC guidance on reporting the effects of an intervention (see: https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources‐for‐authors2017/how_to_report_the_effects_of_an_intervention.pdf), we used standardized statements to indicate the certainty of the evidence to readers. In this approach, ‘probably’ is used to flag moderate certainty evidence of effect while ‘may’ is used to indicate low certainty evidence of effect. As the evidence for a number of outcomes in this review was graded as moderate certainty, the term ‘probably’ was used frequently.

Response to Comment #2: The review is written by health research scientists, with different specializations in epidemiology and health promotion sciences. Four authors (ML, MvdB, NW, AvV) have their roots in nursing (bachelor nursing), but have not been in practice for a long time. One author is a medical doctor (KW) and one author (EK) is a statistician.

Contributors

Comments submitted by: Sandra Allison

Response to comments from: Miranda Laurant and Anneke van Vught on behalf of all authors

Comments submitted via Cochrane Library, 27 July 2018

Summary

Comment #1:

I have issue with the inclusion of several articles in this review. Specifically, multiple articles reference specific populations (e.g. those receiving anti‐retrovirals, those who have rheumatologic conditions). I do not think the experience of these patients can be generalized to the whole primary care population

Comment #2:

Multiple studies are more than 18 years old ‐ some are 45 years old. I do not think the care and experience of care from 45 yrs ago compares to the standards of care today.

Comment #3:

Some studies reference nurses and some reference nurse practitioners, who have very different scopes of practice and training.

Comment #4:

I have respect and admiration for my nursing and nurse practitioner colleagues and am heartened to hear that patients feel well taken care of and are well taken care of in primary care settings. I'm concerned that this article could lead health policy specialists and governments to believe that a nurse is equal to or better than a physician at providing all kinds of primary care.

Family physician training (in Canada) has emphasized patient oriented, full person care in a biopsychosocial model. The old school model of the "general practitioner" has been replaced by a minimum two year post‐graduate training degree in "Family Medicine". I'm concerned that this review doesn't reflect the intricacies and value of my expertise as a generalist Family Physician, as it lumps the care of specialized populations and/or populations from a different era with the population presenting for primary care today

Reply

Response to Comment #1:

The criteria for considering studies for this review focused on primary healthcare service settings that provide first contact and ongoing care for patients with all types of physical health problems. Whether or not the patient population in a particular trial was representative of the whole primary care population was not a selection criteria.

In response to your comments, we once again critically examined whether all of our included studies met our inclusion criteria. We have now decided to exclude two studies that on closer inspection do not appear to have been conducted in primary care settings. These studies focus on care for patients with rheumatologic conditions (i.e. Ndosi 2013 and Larsson 2014). The review will be amended by excluding these two studies and a minor update published in the second quarter of 2019.

The other studies that focus on specific populations of service users are clearly studies undertaken in primary care. In the next update of this review, we will consider in the analyses and interpretation whether the nurses in the included studies provided more generalist or more specific types of care, and try to explore the applicability of the results to these different types of care.

Response to Comment #2:

Year of study/publication was not an exclusion criteria for this review, although we have considered this and will do so again when the next update of the review is undertaken. We decided not to exclude studies on the basis of year of study/publication as primary healthcare is organized very differently worldwide and even though some studies were conducted some time ago, their results may still be relevant to some primary healthcare settings practices today. It is also worth noting that most of the included studies were published from 2010 onwards. In future updates, we will consider the age of the studies when discussing the interpretation of the results

Response to Comment #3:

We agree that the training of nurses, registered nurses, practice nurses and nurse practitioners is different worldwide and, in addition, the terms used to describe nurses with different types of training are not consistent from setting to setting. Indeed, there is considerable variations between, and sometimes within, countries regarding the training and role of nurses who use the title ‘Nurse Practitioners’ (see, amongst others, Maier 20151 and Maier et al, 20162). Further, it is often difficult to assess the training received by nurses in the studies included in the review.

In this review, however, we were interested in situations in which task(s) formerly performed by physicians were transferred to nurses (i.e. substitution of care), regardless of the training or scope of practice of these nurses. We planned a subgroup analyses based on the level of nursing education to explore the issue that you raise. However, we were not able to perform this due to inadequate data, so it remains uncertain how level of nursing education impacts on outcomes when nurses are substituted for doctors in primary care.

1Maier CB. The role of governance in implementing task‐shifting from physicians to nurses in advanced roles in Europe, U.S., Canada, New Zealand and Australia. Health Policy. 2015; 119: (12), 1627‐1635.

2Maier CB, Barnes H, Aiken LH, Busse R. Descriptive, cross‐country analysis of nurse practitioners in size countries: size, growth, physician substitution potential. BMJ Open. 2016; 6 (9): e011901.

Response to Comment #4:

We understand your concerns and, like all Cochrane reviews, this review does not make any recommendations regarding the replacement of primary healthcare doctors with nurses. Such decisions need to be taken within a specific context, and will involve a range of other evidence and information in addition to evidence of the effectiveness of nurses as substitutes for doctors in primary care. Overall, the review shows that ‘trained nurses, such as nurse practitioners, practice nurses, and registered nurses, probably provide care that is equal to or of better quality than that provided by primary care doctors, and probably achieves equal or better health outcomes for patients.’ However, we also note that we cannot conclude whether it is better to deploy nurses providing care for a broad range of health issues or nurses who target groups of patients, and that we cannot draw conclusions on the level of nursing education that leads to the best outcomes when nurses are substituted for doctors. In addition, the certainty of the available evidence varies – the likelihood that the effect will be substantially different from that found in the review is moderate or high for some findings. Nonetheless, the outcomes all pointed in the same direction, towards at least equal care and patient outcomes when nurses are substituted for doctors.

We would suggest that any new models of primary healthcare, including nurses in independent practice roles or substituting for physicians in other ways, should be considered by policy makers in dialogue with doctors, nurses, and other professionals, patients and the public to ensure the quality of, and access to, primary healthcare and to ensure that these models are acceptable and feasible.

Contributors

Comments submitted by: Oona Hayes

Response to comments from: Miranda Laurant and Anneke van Vught on behalf of all authors

What's new

History

Protocol first published: Issue 4, 1998
Review first published: Issue 2, 2005

Notes

This is an update of the review “Substitution of doctors by nurses in primary care”, which was first published in 2005 for the Cochrane Library (Laurant 2005).

Acknowledgements

We thank the members of Cochrane Effective Practice and Organisation of Care for their valuable input during the update of this review. The Norwegian Satellite of the EPOC Group receives funding from the Norwegian Agency for Development Cooperation (Norad), via the Norwegian Institute of Public Health, to support review authors in the production of their reviews. We would like to thank Jan Odgaard‐Jenssen for his guidance, advice, and assistance with statistical meta‐analyses and sensitivity analyses. We thank Marit Johnson who advised on the search strategy and helped with computerised searches, as reported in this update. We would also like to acknowledge Simon Lewin for his appreciated editorial input into the updated review, as well as the following editors and peer referees, who provided comments to improve the protocol: Claire Glenton, Jane Noyes, Connor Emdin, and Claudia Maier; and to Dolores P. Matthews for copyediting this review.

We would like to acknowledge the following people, who were involved in the original review: Steve Rose and Richard Heywood, who advised on the search strategy and helped with computerised searches (original searches until 2002); Michelle Sergison and Bonnie Sibbald, who helped design the review and assisted with early data extraction; and Shirley Halliwell, who obtained study articles and helped to screen papers suitable for data extraction; David Reeves, who helped with statistical analysis in the original review (published in 2005). Finally, we have removed David Reeves, Rosella Hermens, Joze Braspennig, and Richard Grol as authors of this updated review, as owing to other obligations they lack sufficient time to commit to the process of this update. With regret, we have also removed Bonnie Sibbald from the list of review authors, as Bonnie was (together with Michelle Sergison and Miranda Laurant) one of the initiators of this systematic review and had been intensively involved in the review update until the time of her retirement. We would like to thank Bonnie for all of her input into this review until now.

The Effective Health Care Research Consortium provided funding to make this review open access. The Consortium is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this review do not necessarily reflect UK government policy.

Appendices

Appendix 1. Search strategies 2015

CENTRAL, the Cochrane Library (searched 2017)

CENTRAL, the Cochrane Library (searched 2015)

MEDLINE, Ovid (searched 2017)

MEDLINE, Ovid (searched 2015)

CINAHL, EbscoHost (searched 2017)

CINAHL, EbscoHost (searched 2015)

Open Grey = 21 hits (27.02.2017)

1. ("nurse led" OR "nurse managed" OR "nurse run" OR "nurse delivered") = 18 hits

2. (substitute OR substitution OR substituting) AND (nurse OR nurses) AND (doctor OR doctors OR physician OR physicians OR "general practitioner" OR "general practitioners") = 3 hits

Grey Literature Report = 97 hits (27.02.2017)

The Grey Literature Report was searched on 27 February 2017 using different search terms:

1. "nurse practitioner" = 14 hits

2. "nurse clinician" = 1 hit

3. "nurse led" = 6 hits

4. "nurse managed" = 65 hits

6. "nurse delivered" = 11 hits

7. substitut ‐ with Additional Keywords: doctor = 0 hits

8. substitut ‐ with Additional Keywords: physician = 0 hits

9. substitut ‐ with Additional Keywords: "general practitioner" = 0 hits

International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO): www.who.int/ictrp/en/= 71 hits (21.02.2017)

1. Advanced search: nurse led OR nurse managed OR nurse run OR nurse delivered [in Title + Recruitment status: All]

OR

nurse led OR nurse managed OR nurse run OR nurse delivered [in Intervention + Recruitment status: All]

2. Advanced search: substitut* AND nurse* AND (doctor* OR physician* OR general practitioner OR general practitioners) [in Title + Recruitment status: All]

OR

substitut* AND nurse* AND (doctor* OR physician* OR general practitioner OR general practitioners) [in Intervention + Recruitment status: All]

ClinicalTrials.gov, US National Institutes of Health (NIH):clinicaltrials.gov/= 172 hits (21.02.2017)

1. Search Terms: "nurse led" OR "nurse managed" OR "nurse run" OR "nurse delivered"

2. Search Terms: (substitute OR substitution OR substituting) AND (nurse OR nurses) AND (doctor OR doctors OR physician OR physicians OR "general practitioner" OR "general practitioners")

Science Citation Index and the Social Sciences Citation Index, Web of Science, Thomson Reuters = 41 hits (2015)

Citation search for the following studies: Campbell 2014; Houweling 2011; Iglesias 2013; Larsson 2014; Ndosi 2014

Appendix 2. GRADE profiles

Assessing the certainty [1]of evidence across studies for an outcome

Comparison nurse ‐ doctor substitution in primary care

Certainty assessment of evidence for each outcome

[1] This can also be referred to as ‘quality of the evidence’ or ‘confidence in the estimate’. The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect; i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different”, we mean a large enough difference that it might affect a decision.

[2] Indirectness includes consideration of:

· Indirect (between‐study) comparisons

· Indirect (surrogate) outcomes

· Applicability (study populations, interventions, or comparisons that are different from those of interest).

[3] Other considerations for downgrading include publication bias. Other considerations for upgrading include a strong association with no plausible confounders, a dose response relationship, and if all plausible confounders or biases would decrease the size of the effect (if there is evidence of an effect), or increase it if there is evidence of no harmful effect (safety).

[4]

4 High = This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different** is low.

3 Moderate = This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different** is moderate.

2 Low = This research provides some indication of the likely effect. However, the likelihood that it will be substantially different** is high.

1 Very low = This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different** is very high.

** Substantially different = a large enough difference that it might affect a decision.

Notes

Edited (no change to conclusions)

Data and analyses

Comparison 1

Doctor‐nurse substitution study results

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Characteristics of excluded studies [ordered by study ID]

Characteristics of studies awaiting assessment [ordered by study ID]

Differences between protocol and review

Because of the large number of available randomised trials, we excluded non‐randomised trials from this review (Laurant 2000).

In the next update, we will consider again (1) including subgroup analyses on differently trained nurses related to the level of training of participating nurses; and (2) pooling cost data from studies that reported costs.

We performed no statistical testing for funnel plot asymmetry, as none of the pooled outcomes included more than 10 studies. If more than 10 studies would be included, we will follow the recommendations provided in the Cochrane Handbook for Systematic Reviews of Interventions (Chapter 10.4.3.1.).

Change in authorship: We have added Anneke van Vught, Mieken van der Biezen, Nancy Wijers, Kanokwaroon Watananirun, and Evangelos Kontopantelis to the review author list. We have removed D. Reeves, R. Hermes, J. Braspenning, R. Grol, and B. Sibbald from the list of review authors (see Acknowledgements).

Contributions of authors

This review update was led by AvV and ML. ML, MvdB, NW, and AvV assessed studies for inclusion. ML, MvdB, NW, and AvV participated in data extraction and contributed to data analysis. AvV undertook the meta‐analysis with assistance from the other review authors. ML, MvdB, and AvV drafted the review, drawing on contributions from several review authors, and all review authors commented on this draft.

Sources of support

Internal sources

• Centre for Quality of Care Research (WOK), University of Nijmegen, Netherlands.

• National Primary Care Research and Development Centre (NPCRDC), University of Manchester, UK.

External sources

• Ministry of Health, Welfare and Sports, Netherlands.

• Department of Health, UK.

• The Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries, UK.

Declarations of interest

Miranda Laurant: none known.

Mieke van der Biezen: none known.

Nancy Wijers: none known.

Kanokwaroon Watananirun: none known.

Evangelos Kontopantelis: none known.

Anneke JAH van Vught: none known.

References

References to studies included in this review

Campbell 2014 {published data only}

• Calitri R, Warren FC, Wheeler B, Chaplin K, Fletcher E, Murdoch J, et al. Distance from practice moderates the relationship between patient management involving nurse telephone triage consulting and patient satisfaction with care. Health & Place 2015;34:92‐96. [PubMed] [Google Scholar]
• Campbell JL, Fletcher E, Britten N, Green C, Holt T, Lattimer V, et al. The clinical effectiveness and cost‐effectiveness of telephone triage for managing same‐day consultation requests in general practice: a cluster randomised controlled trial comparing general practitioner‐led and nurse‐led management systems with usual care (the ESTEEM trial). Health Technology Assessment (Winchester, England) 2015;19:1‐212, vii‐viii. [PMC free article] [PubMed] [Google Scholar]
• Campbell JL, Fletcher E, Britten N, Green C, Holt TA, Lattimer V, et al. Telephone triage for management of same‐day consultation requests in general practice (the ESTEEM trial): a cluster randomised controlled trial and cost‐consequence analysis. The Lancet 2014;384:1859‐68. [DOI: ] [PubMed] [Google Scholar]
• Holt TA, Fletcher E, Warren F, Richards S, Salisbury C, Calitri R, et al. Telephone triage systems in UK general practice: analysis of consultation duration during the index day in a pragmatic randomised controlled trial. British Journal of General Practice 2016;66:e214‐8. [PMC free article] [PubMed] [Google Scholar]
• Warren FC, Calitri R, Fletcher E, Varley A, Holt TA, Lattimer V, et al. Exploring demographic and lifestyle associations with patient experience following telephone triage by a primary care doctor or nurse: secondary analyses from a cluster randomised controlled trial. BMJ Quality & Safety 2015;24:572‐82. [PMC free article] [PubMed] [Google Scholar]

Chambers 1978 {published data only}

• Chambers LW, West AE. St John's randomized trial of the family practice nurse: health outcomes of patients. International Journal of Epidemiology 1978;7(2):153‐61. [DOI: 10.1093/ije/7.2.153] [PubMed] [CrossRef] [Google Scholar]

Chan 2009 {published data only}

• Chan D, Harris S, Roderick P, Brown D, Patel P. A randomised controlled trial of structured nurse‐led outpatient clinic follow‐up for dyspeptic patients after direct access gastroscopy. BMC Gastroenterology 2009;9:12. [DOI: 10.1186/1471-230X-9-12] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Dierick‐van Daele 2009 {published data only}

• Dierick‐van Daele AT, Metsemakers JF, Derckx EW, Spreeuwenberg C, Vrijhoef HJ. Nurse practitioners substituting for general practitioners: randomized controlled trial. Journal of Advanced Nursing 2009;65(2):391‐401. [DOI: 10.1111/j.1365-2648.2008.04888.x] [PubMed] [CrossRef] [Google Scholar]
• Dierick‐van Daele AT, Steuten LM, Metsemakers JF, Derckx EW, Spreeuwenberg C, Vrijhoef HJ. Economic evaluation of nurse practitioners versus GPS in treating common conditions. British Journal of General Practice 2010;60(570):e28‐e35. [DOI: ] [PMC free article] [PubMed] [Google Scholar]

Hemani 1999 {published data only}

• Hemani A, Rastegar DA, Hill C, Al‐Ibrahim MS. A comparison of resource utilization in nurse practitioners and physicians. Effective Clinical Practice 1999;2(6):258‐65. [PUBMED: 10788023] [PubMed] [Google Scholar]

Houweling 2011 {published data only}

• Houweling ST, Kleefstra N, Hateren KJJ, Groenier KH, Meyboomde Jong B, Bilo HJG. Can diabetes management be safely transferred to practice nurses in a primary care setting? A randomised controlled trial. Journal of Clinical Nursing 2011;20:1264–72. [DOI: 10.1111/jan.12120] [PubMed] [CrossRef] [Google Scholar]

Iglesias 2013 {published data only}

• Iglesias B, Ramos F, Serrano B, Fàbregas M, Sánchez C, Garcíam J, et al. A randomized controlled trial of nurses vs. doctors in the resolution of acute disease of low complexity in primary care. Journal of Advanced Nursing 2013;69(11):2446–57. [DOI: 10.1111/jan.12120] [PubMed] [CrossRef] [Google Scholar]

Larsson 2014 {published data only}

• Larsson I, Fridlund B, Arvidsson B, Teleman A, Bergman S. Randomized controlled trial of a nurse‐led rheumatology clinic for monitoring biological therapy. Journal of Advanced Nursing 2013;70(1):164–75. [DOI: 10.1111/jan.12183] [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• Larsson I, Fridlund B, Arvidsson B, Teleman A, Svedberg P, Bergman S. A nurse‐led rheumatology clinic versus rheumatologist‐led clinic in monitoring of patients with chronic inflammatory arthritis undergoing biological therapy: a cost comparison study in a randomised controlled trial. BMC Mucoloskeletal Disorders 2015;16:354. [DOI: 10.1186/s12891-015-0817-6] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Lattimer 1998 {published data only}

• Lattimer V, George S, Thompson F, Thomas E, Mulle M, Turnbull J, et al (the South Wiltshire Out of Hours Project (SWOOP) Group). Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial. BMJ 1998;317:1054‐9. [DOI: 10.1136/bmj.317.7165.1054] [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• Lattimer V, Sassi F, George S, Moore M, Turnbull J, Mullee M. Cost analysis of nurse telephone consultation in out of hours primary care: evidence from a randomised controlled trial. BMJ 2000;320:1053‐7. [DOI: 10.1136/bmj.320.7241.1053] [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• South Wiltshire Out of Hours Project (SWOOP) Group. Nurse telephone triage in out of hours primary care: a pilot study. BMJ 1997;314:198‐9. [PUBMED: 9022437] [PMC free article] [PubMed] [Google Scholar]
• Thompson F, George S, Lattimer V, Smith H, Moore M, Turnbull J, et al. Overnight calls in primary care: randomised controlled trial of management using nurse telephone consultation. BMJ 1999;319:1408. [DOI: 10.1136/bmj.319.7222.1408] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Lewis 1967 {published data only}

• Lewis CE, Resnik BA. Nurse clinics and progressive ambulatory patient care. New England Journal of Medicine 1967;277:1236‐41. [DOI: 10.1056/NEJM196712072772305] [PubMed] [CrossRef] [Google Scholar]

Moher 2001 {published data only}

• Moher M, Yudkin P, Wright L, Turner R, Fuller A, Schofield T, et al. for the Assessment of Implementation Strategies (ASSIST) Trial Collaborative Group. Cluster randomised controlled trial to compare three methods of promoting secondary prevention of coronary heart disease in primary care. BMJ 2001;322:1338‐42. [DOI: 10.1136/bmj.322.7298.1338] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Mundinger 2000 {published data only}

• Lenz ER, Mundinger MO, Hopkins SC, Lin SX, Smolowitz JL. Diabetes care processes and outcomes in patients treated by nurse practitioners or physicians. The Diabetes Educator 2002;28(4):566‐97. [PubMed] [Google Scholar]
• Lenz ER, Mundinger MO, Kane RL, Hopkins SC, Lin SX. Primary care outcomes in patients treated by nurse practitioners or physicians: two‐year follow‐up. Medical Care Research and Review 2004;61:332. [DOI: 10.1177/1077558704266821] [PubMed] [CrossRef] [Google Scholar]
• Mundinger MO, Kane Rl, Lenz ER, Totten AM, Tsji WY, Cleary PD, et al. Primary care outcomes in patients treated by nurse practitioners or physicians. A randomized trial. JAMA 2000;283(1):59‐68. [DOI: 10.1001/jama.283.1.59] [PubMed] [CrossRef] [Google Scholar]
• Sox HC. Independent primary care practice by nurse practitioners. [editorial]. JAMA 2000;283(1):106‐8. [DOI: 10.1001/jama.283.1.106] [PubMed] [CrossRef] [Google Scholar]

Ndosi 2013 {published data only}

• Ndosi M, Lewis M, Hale C, Quinn H, Ryan S, Emery P, et al. The outcome and cost‐effectiveness of nurse‐led care in people with rheumatoid arthritis: a multicentre randomised controlled trial. Annals of the Rheumatic Diseases 2014;73:1975‐82. [DOI: 10.1136/annrheumdis-2013-203403] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Sanne 2010 {published data only}

• Sanne I, Orrell C, Fox MP, Conradie F, Ive P, Zeinecker J, et al. CIPRA‐SA Study Team. Nurse versus doctor management of HIV‐infected patients receiving antiretroviral therapy (CIPRA‐SA): a randomised non‐inferiority trial. The Lancet 2010;376(9734):33‐40. [DOI: 10.1016/S0140-6736(10)60894-X] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Shum 2000 {published data only}

• Shum C, Humphreys A, Wheeler D, Cochrane MA, Skoda S, Clement S. Nurse management of patients with minor illnesses in general practice: multicentre randomised controlled trial. BMJ 2000;320:1038‐43. [DOI: 10.1136/bmj.320.7241.1038] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Spitzer 1973 {published data only}

• Sackett DL, Spitzer WO, Gent M, Roberts RS (in collaboration with: Hay WI, Lefroy GM, Sweeny GP. Vandervlist I, Sibley JC, Chambers LW, Goldsmith CH. Macpherson AS, McAuley RG). The Burlington randomized trial of the nurse practitioner: health outcomes of patients. Annals of Internal Medicine 1974;80:137‐42. [DOI: 10.7326/0003-4819-80-2-137] [PubMed] [CrossRef] [Google Scholar]
• Spitzer WO, Kergin DJ. Nurse practitioners in primary care. I. The McMaster University educational program. Canadian Medical Association Journal 1973;108:991‐5. [PUBMED: 1941343] [PMC free article] [PubMed] [Google Scholar]
• Spitzer WO, Roberts RS, Delmore T. Nurse practitioners in primary care. V. Development of the utilization and financial index to measure effects of their deployment. Canadian Medical Association Journal 1976;114:1099‐102. [PUBMED: 1957163] [PMC free article] [PubMed] [Google Scholar]
• Spitzer WO, Roberts RS, Delmore T. Nurse practitioners in primary care. VI. Assessment of their deployment with the utilization and financial index. Canadian Medical Association Journal 1976;114:1103‐8. [PUBMED: 1957144] [PMC free article] [PubMed] [Google Scholar]
• Sweeny GP, Ian Hay WI. The Burlington experience: a study of nurse practitioners in family practice. Canadian Family Physician 1973;19:101‐10. [PUBMED: 2371123] [PMC free article] [PubMed] [Google Scholar]

Venning 2000 {published data only}

• Venning P, Durie A, Roland M, Roberts C, Leese B. Randomised controlled trail comparing cost effectiveness of general practitioners and nurse practitioners in primary care. BMJ 2000;320:1048‐53. [DOI: 10.1136/bmj.320.7241.1048] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Voogdt‐Pruis 2010 {published data only}

• Voogdt‐Pruis H, Beusmans G, Gorgels A, Ree J. Adherence to a guideline on cardiovascular prevention: a comparison between general practitioners and practice nurses. International Journal of Nursing Studies 2010;48(7):798‐807. [DOI: 10.1016/j.ijnurstu.2010.11.008] [PubMed] [CrossRef] [Google Scholar]
• Voogdt‐Pruis HR, Beusmans GH, Gorgels AP, Kester AD, Ree JW. Effectiveness of nurse‐delivered cardiovascular risk management in primary care: a randomised trial. British Journal of General Practice 2010;60(570):40‐6. [DOI: 10.3399/bjgp10X482095] [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• Voogdt‐Pruis HR, Gorgels AP, Ree JW, Hoef EFM, Beusmans GH. Patient perceptions of nurse‐delivered cardiovascular prevention: cross‐sectional survey within a randomised trial. International Journal of Nursing Studies 2010;47:1237‐44. [DOI: 10.1016/j.ijnurstu.2010.02.013] [PubMed] [CrossRef] [Google Scholar]

References to studies excluded from this review

Chambers 1977 {published data only}

• Black DP, Riddle RJ, Sampson E. Pilot project: the family practice nurse in a Newfoundland rural area. Canadian Medical Association Journal 1976;114:945. [PMC free article] [PubMed] [Google Scholar]
• Chambers LW, Bruce‐Lockhart P, Black DP, Sampson E, Burke M. A controlled trail of the impact of the family practice nurse on volume, quality and cost of rural health services. Medical Care 1977;XV(12):971‐81. [PubMed] [Google Scholar]

Flynn 1974 {published data only}

• Flynn BC. The effectiveness of nurse clinicans service delivery. American Journal of Public Health 1974;64(6):604‐11. [DOI: 10.2105/AJPH.64.6.604] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Gordon 1974 {published data only}

• Gordon DW. Health maintenance service: ambulatory patient care in the general medical clinic. Medical Care 1974;XII(8):648‐58. [PubMed] [Google Scholar]

Irewall 2015 {published data only}

• Irewall AL, Ogren J, Bergstrom L, Laurell K, Soderstrom L, Mooe T. Nurse‐led, telephone‐based, secondary preventive follow‐up after stroke or transient ischemic attack improves blood pressure and LDL cholesterol: results from the first 12 months of the randomized, controlled NAILED stroke risk factor trial. PloS One 2015;10(10):(no pagination). [PMC free article] [PubMed] [Google Scholar]

Kinnersley 2000 {published data only}

• Kinnersley P, Anderson E, Parry K, Clement J, Arcarhd L, Turton P, et al. Randomised controlled trial of nurse practitioner versus general practitioner care for patients requesting "same day" consultations in primary care. BMJ 2000;320:1043‐8. [DOI: 10.1136/bmj.320.7241.1043] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Kuethe 2011 {published data only}

• Kuethe M, Vaessen‐Verberne A, Mulder P, Bindels P, Aalderen W. Paediatricasthma outpatient care by asthma nurse, paediatrician or general practitioner:Randomised controlled trial with two‐year follow‐up. Primary Care Respiratory Journal 2011;20:84–91. [PMC free article] [PubMed] [Google Scholar]

McIntosh 1997 {published data only}

• McIntosh MC, Leigh G, Baldwin NJ, Marmulak J. Reducing alcohol consumption. Comparing three brief methods in family practice. Canadian Family Physician 1997;43:1959‐67. [PUBMED: 2255191] [PMC free article] [PubMed] [Google Scholar]

Myers 1997 {published data only}

• Myers PC, Lenci B, Sheldon MG. A nurse practitioner as the first point of contact for urgent medical problems in a general practice setting. Family Practice 1997;14(6):492‐7. [PubMed] [Google Scholar]

Stein 1974 {published data only}

• Stein GH. The use of a nurse practitioner in the management of patients with diabetes mellitus. Medical Care 1974;XII(10):885‐90. [PUBMED: 4437220] [PubMed] [Google Scholar]

References to studies awaiting assessment

Lewis 2016 {published data only}

• Lewis H, Kunkel J, Axten D, Dalton J, Gardner H, Tippett A, et al. Community nurse‐led initiation of antiviral therapy for chronic hepatitis C in people who inject drugs does not increase uptake of or adherence to treatment. European Journal of Gastroenterology & Hepatology 2016;28:1258‐63. [PubMed] [Google Scholar]

Additional references

Andryukhin 2010

• Andryukhin A, Frolova E, Vaes B, Degryse J. The impact of a nurse‐led care programme on events and physical and psychosocial parameters in patients with heart failure with preserved ejection fraction: a randomized clinical trial in primary care in Russia. European Journal of General Practice 2010;16:205‐14. [DOI: 10.3109/13814788.2010.527938] [PubMed] [CrossRef] [Google Scholar]

Bonsall 2008

• Bonsall K, Cheater FM. What is the impact of advanced primary care nursing roles on patients, nurses and their colleagues? A literature review. International Journal of Nursing Studies 2008;45:1090‐102. [DOI: 10.1016/j.ijnurstu.2007.07.013] [PubMed] [CrossRef] [Google Scholar]

Contandriopoulos 2015

• Contandriopoulos D, Brousselle A, Dubois CA, Perroux M, Beaulieu MD, Brault I, et al. A process‐based framework to guide nurse practitioners integration into primary healthcare teams: results from a logic analysis. BMC Health Services Research 2015;15(1):78. [DOI: 10.1186/s12913-015-0731-5] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Denver 2003

• Denver EA, Barnard M, Woolfson RG, Earle KA. Management of uncontrolled hypertension in a nurse‐led clinic compared with conventional care for patients with type 2 diabetes. Diabetes Care 2003;26(8):2256‐60. [DOI: 10.2337/diacare.26.8.2256] [PubMed] [CrossRef] [Google Scholar]

Du Moulin 2007

• Du Moulin MF, Hamers JP, Paulus A, Berendsen CL, Halfens R. Effects of introducing a specialized nurse in the care of community‐dwelling women suffering from urinary incontinence: a randomized controlled trial. Journal of Wound, Ostomy, & Continence Nursing 2007;34:631–40. [DOI: 10.1097/01.WON.0000299814.98230.13] [PubMed] [CrossRef] [Google Scholar]

Egger 1997

• Egger M, Davey Smith S, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ 1997;315 (7109):629‐34. [PMC free article] [PubMed] [Google Scholar]

EPOC 2017

• Cochrane Effective Practice, Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC Resources for Review Authors 2017. Available at epoc.cochrane.org/epoc‐specific‐resources‐review‐authors.

EQF 2016

• European Qualifications Framework. https://ec.europa.eu/ploteus/content/descriptors‐page (accessed 7 February 2018).

Fairall 2012

• Fairall L, Bachmann MO, Lombard C, Timmerman V, Uebel K, Zwarenstein M, et al. Task shifting of antiretroviral treatment from doctors to primary‐care nurses in South Africa (STRETCH): a pragmatic, parallel, cluster‐randomised trial. Lancet 2012;380:889–98. [DOI: 10.1016/s0140-6736(12)60730-2] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Freund 2015

• Freund T, Everett C, Griffiths P, Naccarella L, Hudon C, Laurant M. Skill mix, roles and remuneration in the primary care workforce: who are the healthcare professionals in the primary care teams across the world?. International Journal of Nursing 2015;52:727‐43. [PubMed] [Google Scholar]

Ginneken 2013

• Ginneken N, Tharyan P, Lewin S, Rao GN, Meera S, Pian J, Chandrashekar S, Patel V. Non‐specialist health worker interventions for the care of mental, neurological and substance‐abuse disorders in low‐ and middle‐income countries. Cochrane Database of Systematic Reviews 2013, Issue 11. [DOI: 10.1002/14651858.CD009149.pub2] [PubMed] [CrossRef] [Google Scholar]

GRADEpro GDT 2015 [Computer program]

• GRADE Working Group, McMaster University. GRADEpro GDT. Hamilton (ON): GRADE Working Group, McMaster University, 2015.

Groenewegen 2015

• Groenewegen P, Heinemann S, Gress S, Schafer W. Primary care practice composition in 34 countries. Health Policy 2015;119(12):1576‐83. [DOI: 10.1016/j.healthpol.2015.08.005] [PubMed] [CrossRef] [Google Scholar]

Hesselink 2004

• Hesselink AE, Penninx BW, Windt DA, Duin BJ, Vries P, Twisk JW, et al. Effectiveness of an education programme by a general practice assistant for asthma and COPD patients: results from a randomised controlled trial. Patient Education and Counseling 2004;55(1):121–8. [DOI: 10.1016/j.pec.2003.08.007] [PubMed] [CrossRef] [Google Scholar]

Higgins 2011

• Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. [www.cochrane‐handbook.org]

Hiss 2007

• Hiss RG, Armbruster BA, Gillard ML, McClure LA. Nurse care manager collaboration with community‐based physicians providing diabetes care: a randomized controlled trial. The Diabetes Educator 2007;33:493–502. [DOI: 10.1177/0145721707301349] [PubMed] [CrossRef] [Google Scholar]

Hollinghurst 2006

• Hollinghurst S, Horrocks S, Anderson E, Salisbury C. Comparing the cost of nurse practitioners and GPs in primary care: modelling economic data from randomised trials. British Journal of General Practice 2006;56(528):530‐5. [PMC free article] [PubMed] [Google Scholar]

Horrocks 2002

• Horrocks S, Anderson E, Salisbury C. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ 2002;324:819‐23. [PMC free article] [PubMed] [Google Scholar]

Jarman 2002

• Jarman B, Hurwitz B, Cook A, Bajekal M, Lee A. Effects of community based nurses specialising in Parkinson’s disease on health outcome and costs: randomised controlled trial. BMJ 2002;324:1072–5. [DOI: 10.1136/bmj.324.7345.1072] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Kernick 2000

• Kernick D, Cox A, Powell R, Reinhold D, Sawkins J, Warin A. A cost consequence study of the impact of a dermatology‐trained practice nurse on the quality of life of primary care patients with eczema and psoriasis. British Journal of General Practice 2000;50:555–8. [PUBMED: 10954937] [PMC free article] [PubMed] [Google Scholar]

Kernick 2002

• Kernick D, Powell R, Reinhold D. A pragmatic randomised controlled trial of an asthma nurse in general practice. Primary Care Respiratory Journal 2002;11:6‐8. [11:6–8. doi:10.1038/pcrj.2002.4] [PMC free article] [PubMed] [Google Scholar]

Kontopantelis 2012

• Kontopantelis E, Reeves D. Performance of statistical methods for meta‐analysis when true study effects are non‐normally distributed: a simulation study.. Statistical Methods in Medical Research 2012;21(4):409‐26. [DOI: 10.1177/0962280210392008] [PubMed] [CrossRef] [Google Scholar]

Kontopantelis 2013

• Kontopantelis E, Springate DA, Reeves D. A re‐analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta‐analyses. PLoS ONE 2013;8(7):e69930. [DOI: 10.1371/journal.pone.0069930] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Kooienga 2015

• Kooienga SA, Carryer JB. Globalization and advancing primary health care nurse practitioner practice. Journal of Nurse Practitioner 2015;11(8):804‐11. [Google Scholar]

Kuethe 2011

• Kuethe M, Vaessen‐Verberne A, Mulder P, Bindels P, Aalderen W. Paediatric asthma outpatient care by asthma nurse, paediatrician or general practitioner: randomised controlled trial with two‐year follow‐up. Primary Care Respiratory Journal 2011;20:84–91. [DOI: 10.4104/pcrj.2011.00003] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Laurant 2009

• Laurant M, Harmsen M, Wollersheim H, Grol R, Faber M, Sibbald B. The impact of nonphysician clinicians: do they improve the quality and cost‐effectiveness of health care services?. Medical Care Research and Review : MCRR 2009;66(6):36S‐89S. [DOI: 10.1177/1077558709346277] [PubMed] [CrossRef] [Google Scholar]

Maier 2016b

• Maier CB, Barnes H, Aiken LH, Busse R. Descriptive, cross‐country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential. BMJ Open 2016;6 (9):e011901. [DOI: 10.1136/bmjopen-2016-011901] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Martin‐Misener 2015

• Martin‐Misener R, Harbman P, Donald F, Reid K, Kilpatrick K, Carter N, et al. Cost‐effectiveness of nurse practitioners in primary and specialised ambulatory care: systematic review. BMJ Open 2015;5(6):e007167. [DOI: 10.1136/bmjopen-2014-00716] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Martínez‐González 2014a

• Martínez‐González NA, Djalali S, Tandjung R, Huber‐Geismann F, Markun S, Wensing M, et al. Substitution of physicians by nurses in primary care: a systematic review and meta‐analysis. BMC Health Services Research 2014;14:214. [PMC free article] [PubMed] [Google Scholar]

Martínez‐González 2014b

• Martínez‐González NA, Tandjung R, Djalali S, Huber‐Geismann F, Markun S, Rosemann T. Effects of physician‐nurse substitution on clinical parameters: a systematic review and meta‐analysis. PLoS One 2014;9(2):e89181. [DOI: 10.1371/journal.pone.0089181] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Martínez‐González 2015a

• Martínez‐González NA, Tandjung R, Djalali S, Rosemann T. The impact of physician‐nurse task shifting in primary care on the course of disease: a systematic review. Human Resources for Health 2015;13(1):55. [DOI: 10.1186/s12960-015-0049-8] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Martínez‐González 2015b

• Martínez‐González NA, Rosemann T, Djalali S, Huber‐Geismann F, Tandjung R. Task‐shifting from physicians to nurses in primary care and its impact on resource utilization: a systematic review and meta‐analysis of randomized controlled trials. Medical Care Research and Review 2015;72(4):395‐418. [DOI: 10.1177/1077558715586297] [PubMed] [CrossRef] [Google Scholar]

Martínez‐González 2015c

• Martínez‐González NA, Rosemann T, Tandjung R, Djalali S. The effect of physician‐nurse substitution in primary care in chronic diseases: a systematic review. Swiss Medical Weekly 2015;145:w14031. [DOI: 10.4414/smw.2015.14031] [PubMed] [CrossRef] [Google Scholar]

Naylor 2010

• Naylor MD, Kurtzman ET. The role of nurse practitioners in reinventing primary care. Health Affairs 2010;29(5):893‐9. [DOI: 10.1377/hlthaff.2010.0440] [PubMed] [CrossRef] [Google Scholar]

Newhouse 2011

• Newhouse RP, Stanik Hutt J, White KM, Johantgen M, Bass EB, Zangaro G, et al. Advanced practice nurse outcomes: 1990‐2008: a systematic review. Nursing Economics 2011;29(5):1‐22. [DOI: ] [PubMed] [Google Scholar]

NHS 2016

• NHS England. General Practice Forward View, 2016. https://www.england.nhs.uk/wp‐content/uploads/2016/04/gpfv.pdf (accessed 20June 2017).

Perloff 2016

• Perloff J, DesRoches CM, Buerhaus P. Comparing the cost of care provided to Medicare beneficiaries assigned to primary care nurse practitioners and physicians. Health Services Research 2016;51(4):1407‐23. [DOI: 10.1111/1475-6773.12425] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Petterson 2012

• Petterson SM, Liaw WR, Philips RL, Rabin DL, Meyers DS, Bazemore AW. Projecting US primary care physician workforce needs: 2010‐2025. Annals of Family Medicine 2012;10:503‐10. [DOI: 10.1370/afm.1431] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Poghosyan 2012

• Poghosyan L, Lucero R, Rauch L, Berkowitz B. Nurse practitioner workforce: a substantial supply of primary care providers. Nursing Economics 2012;30(5):268‐94. [PubMed] [Google Scholar]

Rashidian 2013

• Rashidian A, Shakibazadeh E, Karimi‐Shahanjarini A, Glenton C, Noyes J, Lewin S, et al. Barriers and facilitators to the implementation of doctor‐nurse substitution strategies in primary care: qualitative evidence synthesis. Cochrane Database of Systematic Reviews 2013, Issue 2. [DOI: 10.1002/14651858.CD010412] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

RevMan 2014 [Computer program]

• The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

Richardson 1999

• Richardson G. Identifying, evaluating and implementing cost‐effective skill mix. Journal of Nursing Management 1999;5:265‐70. [PubMed] [Google Scholar]

Roland 2014

• Roland M, Nolte E. The future shape of primary care. British Journal of General Practice 2014;64(619):63‐4. [DOI: 10.3399/bjgp14X676960] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Salter and Kothari 2014

• Salter KL, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state‐of‐the‐art review. Implementation Science 2014;9:115. [DOI: 10.1186/s13012-014-0115-y] [PMC free article] [PubMed] [CrossRef] [Google Scholar]

Savrin 2009

• Savrin C. Growth and development of the nurse practitioner role around the globe. Journal of Pediatric Health Care 2009;23(5):310‐4. [PubMed] [Google Scholar]

Swan 2015

• Swan M, Ferguson S, Chang A, Larson E, Smaldone A. Quality of primary care by advanced practice nurses: a systematic review. International Journal for Quality in Health Care 2015;27(5):396‐404. [DOI: 10.1093/intqhc/mzv054] [PubMed] [CrossRef] [Google Scholar]

WHO 2012

• World Health Organization. WHO recommendations: optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting ‐ OPTIMIZEMNH. Geneva: World Health Organization, 2012. [PubMed]

Wijers 2013

• Wijers N, Burgt R, Laurant M. Nurse Practitioner in acute primary care [Verpleegkundig Specialist biedt kansen.Onderzoeksrapport naar de inzet van de verpleegkundig specialist op de spoedpost in Eindhoven]. The Netherlands. Nijmegen: IQ healthcare, UMC St Radboud; Eindhoven: Stichting KOH 2013.

Winter 1981

• Winter C. Quality Health Care: Patient Assessment. [Master of Science Thesis]. Long Beach, CA: California State University, 1981.

References to other published versions of this review

Laurant 2000

• Laurant M, Sergison M. Sibbald B. Substitution of doctors by nurses in primary care. Cochrane Database of Systematic Reviews 2000, Issue 1. [DOI: 10.1002/14651858.CD001271.pub2] [PubMed] [CrossRef] [Google Scholar]

Laurant 2005

• Laurant M, Reeves D, Hermens R, Braspenning J, Grol R, Sibbald B. Substitution of doctors by nurses in primary care. Cochrane Database of Systematic Reviews 2005, Issue 2. [DOI: 10.1002/14651858.CD001271.pub2] [PubMed] [CrossRef] [Google Scholar]

Articles from The Cochrane Database of Systematic Reviews are provided here courtesy of Wiley