The Department of Health headquarters in Whitehall Source: Wikipedia

The decade to come promises a transformation in healthcare, as the digital revolution allows doctors and other medical professionals to share data about patients, improving their care and advancing research.

Over the next few months, a pair of government initiatives will do much to determine how far and how fast the NHS can do this. A panel led by Fiona Caldicott, of which I am a member, is to report on the balance between protecting patient confidentiality and sharing information to enhance care. Ministers are also to seek the views of the public on changing the NHS Constitution to improve access to patient data for critical medical research. As medical data has such power to deliver better understanding of disease and better patient outcomes, it is important we find the best way of sharing it.

Doctors have long understood that the best possible healthcare is built on the best possible data. As a medical student in the 1970s, I was taught that the foundations of diagnosis and treatment were to take a detailed history and to perform a comprehensive clinical examination. The information that these basic techniques reveal is as fundamental to good practice today as it was then. What has changed is our ability to record it and access it, to the benefit of our patients.

In 1978, when I arrived at Hammersmith Hospital to work with the distinguished clinical pharmacologist Colin Dollery, I saw how this can work. Dollery’s practice was among the first to record the results of a structured questionnaire and examination in a computer database, bringing great improvements to the diagnosis and management of high blood pressure. Accurate records led to accurate therapy, monitored for effectiveness and side effects. Complications of hypertension such as heart, eye and kidney disease were identified early and could be prevented by aggressive control of blood pressure.

As IT has advanced still further, it has become as unacceptable for us to practise medicine with fragmented records, often still kept on different pieces of paper in different parts of the health system, as it would be to practise without the latest drugs and diagnostics. It means ignoring technology that has the power to save lives.

Many of us suffer from chronic diseases, such as diabetes, heart disease and cancer, and many more of us will do so as we age. These conditions need joined-up care in many different settings. Yet our general practitioners still often work with one set of clinical records and nurses in hospitals with another. Each and every hospital we attend, in turn, has a different set of records from the last. Our social care providers have access to none of the above.

Digital technology can provide every one of these healthcare professionals with access to a single, integrated record for every patient. It should also allow individual patients to see the data that is held about them, empowering us to become active partners in our own care.

Medical records, of course, contain private data, which could be carelessly or maliciously disclosed. A parent could coerce a daughter into revealing her contraceptive history. An obsessive fan could gain access to the health records of a celebrity. But it is a fiction to imagine that the haphazard paper chits of old are more private than the modern digital alternative. Paper records have always presented a security risk. As for data loss, arguing that this is best mitigated by holding it in incomplete, inaccurate or inaccessible form seems a curious response.

A better one is to ensure that medical data is shared with approved professionals who can put it to good use, while respecting patient privacy. This group should include not only clinicians involved in patient care, but also approved researchers. Most patients with a chronic disease already support this strongly, because they understand the importance of data in developing better prevention, diagnosis and treatment. It is also important that outdated therapies are abandoned in favour of improved ones. For these things to happen, clinical practices must be constantly monitored for effectiveness and side effects – and that requires data.

Research must be central to healthcare if healthcare is to improve. The government’s proposed changes to the NHS Constitution would embed this principle. The result would be to bring closer to reality a vision of the NHS as a system for providing integrated health and social care at an affordable price.