25 March 2015

Level 6, 80 The Terrace, Wellington 6011
PO Box 10509, The Terrace, Wellington 6143
New Zealand
Telephone: +64 4 384 7635 or 0800 286 801
Email: mcnz@mcnz.org.nz
Website: www.mcnz.org.nz

Better Data – the benefits to the profession and the public
Purpose:
The Medical Council of New Zealand (the Council) is releasing this discussion paper to
generate debate within the profession and wider stakeholders including the public on the
subject of what is the value of performance and outcome data, and how such data promotes
the competence of doctors.
This paper addresses two separate but related issues: The role performance data can play to
assure competence and the public release of any qualitative outcome data.
Good quality performance and outcome data should aid patient choice, inform clinicians,
patients, and administrators and should also form a central component of recertification.
Discussion Document:
Recently the Ombudsman, former Health and Disability Commissioner Professor Ron
Paterson, ruled that a DHB should release surgeon-specific case volume data. This was in
response to a journalist seeking information under the Official Information Act (OIA). The
DHB declined to release the information. Professor Paterson raised the public interest
argument and highlighted the availability of such data in other jurisdictions, especially the
National Health Service in the United Kingdom. The information sought in the case before
the Ombudsman was purely for surgical volume for various procedures for a single named
surgeon. The argument against the OIA request included the point that volume data in
isolation would serve little real value.
The Council is aware of further requests of a similar nature from various media outlets to a
number of DHBs. The Ministry of Health and the Council of Medical Colleges are actively
considering the implications of the recent OIA requests. This raises the questions of what
data we should have and what should happen to any data gathered?
What is the value of accurate performance and outcome data? This question should be
addressed before any data are considered for release, as should the questions of openness
of, and access to, information, and of any risk from the release of such data.
Data can be gathered at individual, service, regional, and national levels. Reliable and
accurate data can perform a number of functions including:
a.
Assessment of quality outcomes.
b.
Identification of areas for improvement.

1
Protecting the public, promoting good medical practice
Te tiaki i te iwi whānui me te whakatairanga pai i te mahi e pā ana ki te taha rongoā

 c.
d.
e.
f.
g.

To inform clinicians and allow relevant benchmarking of services.
To inform patients and allow for better informed choice.
To create a meaningful national dataset of risk and benefit.
To influence planning and investment in health systems.
To influence recertification.

Not all these functions refer to individual clinicians and many can be equally applied to
departments or facilities and ultimately to regional or national data sets.
The Council has a central role in assuring the public that their doctors are competent and fit
to practice. To do this, Council relies heavily on successful participation in, and completion
of, recertification programmes under the broad heading of Continuing Professional
Development (CPD).
The Council’s role may not be to release public data on specific aspects of the practice of
individual doctors, but rather to promote the discussion of the potential benefits of accurate
data as well as to establish principles that should define the framework within which
information is used.
Ultimately, in matters of competence the Council seeks to determine if a doctor has met a
required standard. In relation to recertification the profession and stakeholders have a
major role in defining that standard. The Council has no desire to become a clearing-house
for data – we simply needs to be assured the appropriate standard has been met.
The Council’s strategic direction in relation to recertification is based on doctors receiving
information and feedback on their performance, including areas for improvement in their
practise. Employers also undertake quality assurance and improvement mechanisms
including credentialing and annual performance appraisal. Better data are important to all of
these processes.
It is important to examine the arguments for making more performance and outcome data
publicly available, particularly where it relates to individual clinicians. It needs to serve more
than just a ‘right to know’ purpose. Given the public have little choice in terms of where they
receive publicly funded secondary and tertiary hospital treatment it may be argued that the
release of data could be somewhat meaningless in terms of patient choice.
But this could risk ignoring many of the potential wider benefits of better informing the
public. Done well, publicly available data should be reassuring to the public, it should
encourage better engagement in decision making around service provision, and it should aid
informed choice because local outcome figures will be available. Benchmarking to best
practice would add significantly to the value of the data.
The populations of our DHBs are not homogenous and therefore neither should be the data
we rely on to inform health care decision-making. Again, the complexity of this mandates a
carefully considered approach to the whole question of data acquisition and its use,
including the public release of data.
The Council therefore believes the profession and stakeholders should determine the nature
and context of data to be considered in relation to quality improvement, performance
appraisal, and recertification. Separately, the profession and stakeholders should raise with
the public the value of releasing data to the public, and what form that data would take.

2

 CPD should ideally be influenced by knowledge of personal performance. Quality must be a
foundation of any review or critique of personal performance. Quality can only be
confidently known if it is accurately measured. Once measured, it must also be put in
context.
Done well, data can further improve clinician performance and may be of great value to the
public in raising confidence in their health service. Context is critical – for example, if a
surgeon’s 30 day mortality rate after elective bowel cancer surgery was 10 percent that
surgeon would likely be very concerned, but if in fact the surgeon specialized purely in the
high risk, significantly co-morbid type of cases which have benchmarked mortality rate of 20
percent the public (as well as the surgeon and the hospital) would likely be reassured that
the surgeon was providing a competent service. This is far more informative than simply a
mortality rate with no context.
But what role can data really play in patient choice? This is a very complex issue. Ideally data
should identify the risks and benefits relevant to each individual patient. Risk may exist
because of factors related to the patient, the clinician, and the facility. What is important is
to assess each of these components and if the public desire data, to report on all of these
factors.
Patient factors – comorbidities – are crucial to outcome. There are reliable, validated
information systems in which patient factors can be entered and a risk prediction made in
relation to a number of variables including the risk of various complications, loss of
independence, and even death. The key advantage of such risk prediction systems is context
– the clinician and the patient can talk about specific risks relevant to the individual patient.
In relation to the impact of the clinician and the facility in determining outcome, a riskassessment based system allows comparison of actual outcome against predicted outcome
over time. In other words if actual outcomes are consistently better than predicted then the
risk secondary to the patient’s existing health is being mitigated by quality from both the
clinician and facility. This may be the type of data the public would find of benefit rather
than a simple league table of the incidence of complications or outcomes in the hands of a
particular clinician.
We already have some data publicly available such as waiting times and elective surgical
volumes, immunisation rates, screening rates, time in the emergency department and so
forth, but much of this data are quantitative in nature and do little more than inform the
public about access to a service or uptake of a service.
Context is inseparable from any meaningful debate about data. It is imperative that
clinicians do not “game” by avoiding doing the right thing if the right thing might appear to
worsen the clinician’s individual outcome statistics. There are some treatments that always
carry very high complication rates even in the very best of hands, for example, surgery to
remove a cancer of the head of the pancreas gland. Simply releasing numerical data may in
fact be harmful if it paints an inaccurate picture of performance and outcomes in our health
sector.
Qualitative data, on issues such as the risk of losing independence after a particular
treatment, or the potential benefits in terms of gaining quality of life should be central to
any debate about the value of data. These outcomes, after all, are often the central themes

3

 patients should be informed of prior to making a decision about their health and treatment
options.
Overall the public has a major vested interest in the public health system and the public has
both the right and the responsibility to understand and influence the way health is
delivered. Similarly those who choose treatment in the private sector should expect to know
more about likely outcomes.
Qualitative outcome and performance data will allow clinicians to far better reflect on their
own performance as well as plan for their on-going learning. This is the foundation of
recertification. The publicly reported outcome of recertification should be a statement
noting the clinician meets the required standard.
Clearly some specialties will have the ’luxury’ of more robust data than others –
interventional specialties perhaps being the easiest example where a wealth of quality data
should exist and should be utilised, but there are quality indicators in almost all branches of
medicine.
Clinicians have long accepted they have significant accountabilities and responsibilities in the
arena of clinical outcomes, but perhaps the time has arrived for executive managers to
similarly be accountable predominantly for qualitative outcomes rather than having the
emphasis on quantitative volumes?
Gathered accurately, used correctly and explained well, qualitative data down to individual
clinician level could be of considerable benefit to clinicians, administrators, regulators and
the public. This does not however answer the question of who should have access to the
data? Access should be based on the benefits of such access. These groups may well have
differing needs and differing emphasis in terms of data and how it is reported. The Council
would encourage the profession, managerial colleagues, the Colleges and the public to
engage in meaningful debate around data and how best to use it.

Andrew Connolly
Chairman, Medical Council of New Zealand
chair@mcnz.org.nz

4