When I first came into government in the heady days after 1st May 1997, I imagined a brief, frustrating period of struggle with the bureaucracy before a return to normal life. Some insiders had advised me to wait a couple of years before going into government, on the grounds that such an inexperienced group of ministers would be bound to make a mess of things at first. Others said that few people with a think tank background, like mine, had thrived amid the compromise and backbiting of Whitehall and Westminster.
But I was keen to work in government because I was already familiar with big bureaucracies. I had started my career at the GLC and then worked for the European commission. I also knew many ministers fairly well, having run Gordon Brown's office in the early 1990s, and I had been involved in drawing up Labour's programme, which I thought sensibly limited in ambition but practical enough to make a real difference. The lessons of social democratic hubris had been learned, and a disciplined and personable team was entering government on a wave of public enthusiasm.
I stayed for seven years—far longer than I intended—and was able to observe at close quarters what will come to be seen as one of the more successful governments of recent times, at least in domestic policy (despite immense energy, its international policies have brought thin returns so far). In political life the crucial patterns become apparent only in hindsight, but in what follows I nevertheless try to offer some provisional lessons from my experience.
1. Governments have not become powerless
It is widely assumed that governments have lost power—upwards to a globalised market or Brussels, downwards to the people, or outwards to the private sector and the media. This is one of the reasons why social democratic governments have reined in their ambitions, and I expected to leave government more conscious of its constraints than of its possibilities.
But instead I came away convinced that the perception of powerlessness is an illusion. Strong forces do limit government's room for manoeuvre: global markets and treaties impose limits on economic policy, and the media and business constrain government as much as churches and trade unions did a few decades ago. Yet the basic powers of governments have not diminished. The capacity to tax, for example, remains in rude health. Across the OECD, governments' share in GDP has risen over the past few decades; even the tax take (as opposed to the rates) on profits has gone up. Many of the world's most competitive economies are overseen by relatively big governments. Moreover, governments' ability to deal with problems like pollution and organised crime has been enhanced, not diminished, by globalisation. And while governments have reduced their roles in running economies—the vast bureaucracies that a generation ago were running nationalised industries have melted away—this retreat has been matched by a growing role in health, old age, childhood and security. The idea that governments have become impotent is an illusion, albeit one that can provide a useful alibi.
Aneurin Bevan famously remarked that as his career progressed, real power always seemed to lie just beyond his reach. Yet in retrospect, his career is a good example of just how much power for the good can be exercised by government. I suspect that, looking back, Bevan's successors as health secretary in the 1990s and 2000s will be recognised for the remarkable extent of their power to take ideas from conception to delivery (if anything, sometimes too quickly)—not for the constraints they faced.
2. Trust is the most important asset for any government
John Major's regime was an object lesson in how to lose trust. The dramatic exit from the ERM in 1992 irreparably damaged its reputation for economic competence and was the single most important factor in Labour's 1997 victory. The "back to basics" rhetoric, combined with a succession of low-level scandals, destroyed the Major government's integrity at a time when politicians were already coming to be held in low esteem, derided for unrealistic promises, poor delivery and an inability to tell the truth. George Bush senior's "read my lips" promise not to raise taxes was prominent in the political memory.
Labour learned these lessons well in opposition, and concluded that it should promise only what could be delivered and to show no tolerance of sleaze. Having seen the correlation between people's knowledge of public agencies and the extent to which such bodies were trusted, the party determined to communicate not only what it was doing but also why, providing a running commentary on its own actions.
Yet in government, these lessons were sometimes forgotten. One problem was that the mandate given by the voters in 1997 was bigger than that which had been asked for: the modesty of Labour's promises contrasted with the apparent enthusiasm of the public. And too often, the gap between a modest pledge and the public's raised expectations was filled by grand ministerial rhetoric. On many occasions, government spending announcements were inflated or repeated, with the result that Labour's impressive rises in spending on health and education have been devalued by earlier spin. Some pilot projects that had barely started were proclaimed as successes, and many programmes that depended on the slow graft of building up community capacity—like Sure Start or the neighbourhood regeneration programmes—were implemented too quickly. The famous communications machine turned out to be expert at handling day-to-day rebuttals. But it was less good at changing hearts and minds in the manner of Margaret Thatcher—there is disappointingly little evidence of any shift towards progressive values amongst the British public over the last 8 years—and it was unsuited to building public trust.
This matters because there is nothing inevitable about low trust in government. The public today is certainly less deferential. But in many countries, trust levels have risen, and even in the English-speaking world, where trust has generally fallen, the picture is uneven. For example, in 1975 only 20 per cent of Americans aged 18-29 had confidence in those running the military, yet a quarter of a century later the military was the most trusted US public institution: in 2000, 63 per cent of US citizens had confidence in the military compared to 25 per cent in congress. The reasons included successful performance in Grenada, Panama and the Gulf war, professionalism (on racial integration and drug abuse, for example) and active communication to burnish its image (which may, of course, now be at risk in the wake of Abu Ghraib and Baghdad).
Another example is food. Trust in the British government's ability to secure food safety collapsed after the trauma of BSE. Yet the arrival of the food standards agency, which has worked openly and acknowledges ambiguity, has pushed trust back up. The general lesson is that changes in levels of trust are explained far better by how organisations behave than by larger trends. Institutions that are competent, pursue a clear public moral purpose, deal intensively with the public and quickly admit their errors tend to retain high trust. None of these attributes comes easily to governments.
3. Governments overestimate their power to achieve change in the short term, and underestimate it in the long term
Six years after 1997, the strategy unit was commissioned by the cabinet to conduct a "strategic audit." The aim was to take stock of how the country was doing and how well government was performing. The exercise involved a systematic comparison of Britain against other countries, assessments of what was happening in each important area of policy, and anonymous interviews with almost all cabinet ministers and most of the permanent secretaries.
Taking this long view showed up those areas in which Britain was doing well (economic growth and employment, for example, and CO2 reductions) and those in which we were still underperforming (R&D, productivity, congestion and inequality). The countries doing best on many fronts were the smaller ones of northern Europe, particularly in Scandinavia, rather than the favoured models of the past—the US, Germany, France and Japan. All had found distinctive new ways to combine open economies and political systems with high levels of capacity—particularly human and social capital. The countries that had seen the sharpest improvements over the last decade shared another feature: they had focused on the long term and the strategic. Most of the frontrunners in the latest world competitiveness rankings—Finland (1), Sweden (3), Taiwan (4), Denmark (5), Norway (6), Singapore (7)—contained specialist teams within their bureaucracies whose job was to look at long-term strategy and to challenge complacency.
When Labour came to power in 1997, Whitehall's ability to think and act strategically had atrophied. The central policy review staff (CPRS)—founded by Edward Heath in 1970—had disappeared more than a decade earlier. With the wafer-thin majorities of the mid-1990s, No 10 thought in terms of days rather than decades, and the treasury was little better, scarred by its failure to understand, let alone manage, the rollercoaster cycles of the 1980s and early 1990s.
After 1997, Tony Blair moved steadily to build up capacity at the centre. The social exclusion unit, which I helped to set up, established some of the principles: an emphasis on analysis; an open process with as much work published as possible; a 50:50 split between insiders and outside practitioners; and rapid moves to implement conclusions and follow them through under the aegis of cabinet committees. After 1998, the strategy unit (originally the performance and innovation unit) spread these principles across other policy areas and soon became part of the government machine. It drew most of its commissions from ministers rather than just the prime minister, and was helped by a governance structure that carefully wove together No 10 and No 11.
Over the last few years a quiet revolution has taken place, largely ignored by the media, which are bored by the mechanics of government. Each department has established a strategy team, often run by a senior figure from outside government. The five-year strategies published by all the major departments over the last nine months mark a decisive step towards a more serious approach to the business of government, and have generated much interest around the world, from Brazil and China to Russia and Japan.
Taking a strategic approach is difficult in any government—you rub up against pressures of tactics and politics, and can be undermined by personality clashes. But a combination of sound analysis, rigour on priorities and realism about capacities to deliver does pay dividends. This is evident internationally, but it is also clear from recent British history. The clearest message to emerge from a comparison of the 2003 strategic audit with the similar one conducted under the CPRS in 1971 was that many issues that had once appeared intractable had gone on to be treated or cured. Our predecessors had despaired that problems like high inflation, unemployment and strikes were not amenable to policy. Yet as the Times commented on the strategic audit: "What looks insoluble to one generation can be sorted out more completely than would have been thought possible… but governments overestimate their influence and impact in the short term and underestimate it in the long term."
4. Government must draw on independent knowledge
This partial shift to a more strategic style of government reflects a changed relationship between government and knowledge. Past governments drew mainly on ideology, instinct or political calculation to determine what to do. But now that there is far more evidence on what is likely to work in fields as diverse as penal policy and macroeconomics, the craft of government has become a bit more like a science. This knowledge resides in universities, in international organisations like the OECD or EU, and in government itself. Much of the evidence is banal, but often it can show quite counterintuitive results: that there is little correlation between spending on education and results, for example, or that spending on drugs enforcement usually strengthens organised crime.
Government's greatest successes have generally been in areas in which the knowledge base is strongest and where independent validators of knowledge, like the audit commission, are most powerful. So the decision to pass power to the Bank of England has made it possible for decisions on interest rates to be made openly on the basis of evidence and economic knowledge, with peer review and a remarkable degree of frankness about the uncertainties involved. In social policy, my main focus between 1997 and 2000, almost everything we did rested on a strong knowledge base: the new deal drew on the experience of welfare-to-work programmes in Scandinavia, North America and Australia, many of which had been rigorously evaluated. Sure Start drew on a mountain of evidence about the impact of early years support. Pilot studies designed to generate new knowledge have become commonplace, on the principle that it is generally better to test an idea in a small area rather than on the whole population at once.
It cannot be entirely a coincidence that some of the government's greatest problems have arisen from the field—intelligence—that has been most immune to this gradual revolution. In the past, intelligence agencies have been notorious for skewing secret advice to suit their own interests, usually by exaggerating threats to gullible politicians. In retrospect, despite the often sober peer review of the joint intelligence committee, when it came to judging WMD in Iraq, there was neither enough external scrutiny nor enough rigorous assessment of the status of the knowledge. By contrast, the more open systems for managing knowledge in the UN and the media turned out to be rather better at judging the truth.
This growing emphasis on knowledge does not exclude a role for values or ideals. The knowledge base is usually uneven and no amount of it can tell any government what it should do or what it should value. However, it can steer it away from stupid mistakes and futile efforts. And in some fields, evidence can powerfully reinforce values, as in the case of climate change.
5. Governments have to renew or die
All governments risk stagnation. There are natural cycles of growth and decay. Administrations that start their life fresh and full of zest tend to become stale as politicians start to believe their own propaganda, are trapped by old assumptions and mingle only with sycophants. Yet some administrations have renewed themselves, often over many decades. During Labour's first term, I visited several countries where ruling parties or coalitions had remained in power over long periods, including Canada, Japan, the Netherlands and Sweden, and tried to draw some lessons. These turned out to be fairly simple. First, renewal depended on new people: at some point there had to be wholesale changes of personnel, sometimes including the leader. Nothing better symbolises renewal than a selection of younger faces to replace an old guard. Second, there had to be new stories, new ways of describing what the parties were trying to achieve and why. Third, there had to be new policies which embodied these stories. And fourth, there had to be a new way of communicating, since the methods that originally help a party gain power face a law of diminishing returns (John Major's use of a soapbox in the 1992 election is a good example—the antithesis of Saatchi and Saatchi bombast).
Over the last few years, Labour has attempted to apply these lessons. There has not been much change in personnel, with a few exceptions, such as the rise of Ruth Kelly. But the party has been groping towards a new narrative based around children, empowerment and social mobility, and it has worked hard to renew policy, aware that inadequate policy development before the 2001 election had not properly prepared the ground for the introduction of foundation hospitals and tuition fees. By contrast, the five-year plans published in 2004, which have set out much of Labour's manifesto well in advance of the election, represent a far-reaching renewal of policy. And the Big Conversation exercise represents a modest first step towards a new way of talking to the public.
After only five years in office, the Attlee government was widely seen to have run out of ideas. It is one of the great achievements of both Blair and Brown that they have sustained momentum, and are more often criticised for their excessive zeal.
6. Dynamic governments remain porous
Renewal rarely comes only from within. One of the optical illusions of government is that those inside it think of themselves as the drivers of change. Energetic leaders do cajole, prod and persuade. Yet most far-reaching ideas and changes come from outside, from social movements and movements of ideas. Governments are more often vehicles than initiators. They play a role in embedding these changes but typically they get involved only at a late stage. Consumerism in public services, activism on third world debt and aid and new thinking about child development all evolved either outside government or on its margins before being brought into the mainstream.
This is why it is so important for governments to remain porous—open to the views and ideas of business and NGOs, public servants and the public—and why it is sometimes necessary for even the most powerful politicians to take time out to listen and learn. For the same reasons, the smarter governments around the world realise that they need to build innovation into their everyday working: through experimental zones and pilots, competitive funds and rewards for promising new ideas. And new ideas need time to evolve—preferably away from the spotlight.
For all the talk of control freakery, New Labour's big tent has made for a more conversational style of government in which much more policy is offered in draft for comment and discussed with others than ever before. The strategy unit, for example, has often published its detailed work plans, interim papers and emerging ideas, and encouraged anyone who might be involved in change to take part in the design of policy—a far cry from the unhealthy mix of secrecy and amateurism that characterised so much policymaking in the 1970s and 1980s.
However, in some respects government has been less porous than it should have been. Instincts towards secrecy remain strong, and are reinforced by the threat of leaks to a lurid press (on one memorable day the Times ran six pieces on a spurious story about a project for which I was responsible—not one journalist had bothered to find out if it was accurate). One consequence is that work is often done in small teams without adequate involvement of experts or practitioners, let alone the wider public.
Excessive centralisation also remains a major problem: despite some modest steps, the much-needed revitalisation of local government has scarcely begun. Other countries benefit from having towns, cities and regions with the power and money to experiment and pave the way for new national policies. Their politicians are tested in local politics before they take on national roles, whereas many of ours have never run anything before they are thrown into ministerial roles. More fiscal autonomy and powers for local government, and more political reforms (including elected mayors) would help. But until the main parties encourage their high-flyers to prove themselves in local government rather than rushing to Westminster, little will change.
7. Governments need ideology; a governing philosophy
Most progress involves some change to absolute and relative power. It requires governments to take on vested interests, to use guile and clout in defeating them, and to draw on the energy that comes from a guiding vision and values. In some respects, New Labour has been deliberately non-ideological, emphasising what works and avoiding too sharp an ideological definition for fear of alienating parts of the big tent coalition that it assembled. It came to power not, as in 1945, with a strong consensus behind reform, but in the aftermath of a period in which the self-confident (and self-deluding) ideologues of Thatcherism had roundly defeated their opponents on the left.
Yet New Labour's leaders realised that without a governing philosophy it would be hard to provide coherence to the flotilla of bodies that makes up a modern state. So New Labour has expended much effort seeking to articulate its governing philosophy. The many summits, seminars and publications on the third way are testimony to this. The 2003 progressive governance conference in London, which brought together 13 heads of government and hundreds of politicians and intellectuals from all over the world, had no precedent in British history as an exercise in ideological export.
But the job of ideological redefinition was constrained. Political parties exist above all to win elections, and during the 1980s Labour had become embarrassingly bad at this. The primary designers of New Labour came from communication and marketing backgrounds. They did a fine job of rebranding the party, recasting its messages and policies to better fit the concerns of swing voters. But what was on offer was not an ideology or a strategy of transformation. It was mainly a way of winning elections.
Unfortunately, the very factors that made it a success as an electoral project inevitably weakened it as a transformative governing project. Despite substantial progress in reducing poverty and opening up opportunities, the big tent approach made it hard to take on the most powerful interests—the London media, the super-rich, big business and the City—that often stood in the way of progressive reform. Policies in areas as varied as curriculum reform, environmental regulation and the taxation of pensions for the rich were unnecessarily constrained. As I explain below, this caution also hampered the work of building up new institutions. More subtly, the desire to keep all sides on board meant that there were not enough clear principles to guide civil servants in any given situation, leaving many reliant on the hangovers of the previous regime's philosophy (a generic version of neoclassical economics).
New Labour avoided the worst mistakes of Bill Clinton, whose desire to "triangulate" greatly reduced his long-term impact. But many of the younger people involved in New Labour felt that the older politicians and advisers were too in awe of the Thatcherite legacy, too deferential to neoliberal thinking, too frightened of the press and insufficiently attentive to how much the public had moved on.
Various efforts have been made to provide greater ideological definition without falling into the trap either of defining New Labour in terms of accommodation to a rightwing agenda or of returning to an obsolete leftism. Some of the work has been done in universities and think tanks, some in government. For example, the strategy unit worked on the idea of "public value" to provide a theoretical underpinning for public service reform; it worked on the theory and practice of how to maintain universality and equity alongside greater choice and personalisation in public services; on the nature and extent of personal responsibility; on the role of government in advancing happiness (taken up by Richard Layard); and the policy options for social mobility or poverty. But there has probably been less appetite for theoretical work than in previous periods of Labour or Tory dominance.
I suspect that if ministers had their time again, this is one thing they might change. More ideological clarity might have carried some electoral cost; but it would have given government more edge and impact and would have kept the party more united.
8. All ideas have to be embodied in organisations
We live in a world of organisations; ideas and values that do not take root in organisations tend to wither. This is why institution-building is essential for any political party concerned about radical change. Both Labour after 1945 and the Tories after 1979 understood that new habits and values had to be embedded in institutions.
In some respects Labour has been radical about institutions: it has implemented the most far-reaching constitutional changes for many generations; it has created new regional bodies and a host of new public services, like LearnDirect and Connexions. It recognised that the existing departments and agencies do not work well in dealing with issues like entrenched poverty, and that top-down structures do not respond well to more demanding citizens. It also pioneered moves beyond classic 19th-century administration based on departments towards something a bit more like joined-up government, with cross-cutting budgets, and a more flexible mix of temporary project teams and permanent structures.
But in retrospect, New Labour did not go far enough. Its leaders had little experience of running organisations and tended to believe that if only you put the right people in charge, everything would be fine. Whenever there was a clash between the old forms of power based on the major public professions (doctors, teachers, police) and emergent new forms of power, the old tended to win, helped by their champions in Whitehall. The result is that despite some useful experiments, Britain still awaits a radical reformer who can recast the state to cope better with big issues like environmental change, poverty or localism.
The experience of grappling with institutional change transformed how I thought about my own career. When I had been in government for two and a half years, Michael Young asked if I would take over from him as head of the Institute of Community Studies. The ICS was the small vehicle through which Young had done research and helped to create many dozens of organisations like the Consumers' Association (1956), the Social Science Research Council, now the ESRC (1965), and the Open University (1969). Some, like the School for Social Entrepreneurs and Grandparents Plus, he launched as recently as the late 1990s. He also paved the way for many of the major reforms of recent years—from NHS Direct to after-school clubs. His insight was that if ideas were embedded in new organisations, they could often achieve a more lasting impact than policies and legislation. In his subtle way he said to me that although it was important to know how government worked, more lasting change could sometimes be achieved from outside.
I decided to stay for a few more years in government, even as I argued strongly for the principle, introduced last year, that senior civil servants should not stay in the same job for more than four years. But by the time Michael died in 2002 I had concluded that he was right, that most radical change has to start from outside government, and usually from the bottom rather than the top. Government has a crucial role to play: but it often comes later. That was why I left in 2004 and am now building up the Young Foundation, based on Michael's legacy, and devoted to a mix of research and institution-building.
Political reform does not happen easily. It depends on analysis, strategy and implementation, as well as luck. But one other thing I learned from Michael was that beneath the surface it often depends even more on the commitment and energy of quite small groups. All of the more radical recent reforms were driven forward by tightly-knit networks, rarely more than half a dozen people, including a few key ministers, advisers, civil servants and outsiders. When such a group has a clear understanding of what it is trying to achieve, a huge amount can be done. In departments where such an alignment was lacking there might be lots of activity and announcements—but the whole added up to less than the sum of its parts. Historical accounts of the drive to civil service or educational reform in mid-19th century Britain, or health reform in the middle of the 20th century, tell similar stories of small groups having disproportionate effects through a mix of high vision and low cunning, formal structures and informal networks, and a melding of a passion for change with dispassionate realism about how it can be done.
Many leave government disillusioned about its ability to achieve change and cynical about politicians' motivations. I left with rather opposite lessons. Government retains an enormous power for good, and the vast majority of ministers and officials I have observed are just as impressive, and ethical, as their counterparts in any other sector. Like any human venture, government can be full of error, fallibility and hubris. But the bigger danger for governments today is not excessive hubris but rather that they might succumb to the myth—often propagated by a sceptical media—that they are powerless, condemned to mistrust and futility. If they do so succumb, they will fail to rise to the great challenges, from climate change to inequality, that they alone can tackle.
