Ball in his court: Johnson squares up to his old rival David Cameron in Trafalgar Square for International Paralympic Day, 8th September

“This job is the most engrossing I’ve had,” Boris Johnson told me in early September. London’s mayor acknowledged that it took “a year or so” for him to appreciate the role, after beating Ken Livingstone in 2008. As mayor of the capital, directly elected (unlike the prime minister or cabinet members) by nearly 1.2m people, Johnson has the biggest mandate of any politician in the country. Recent polls show he is set to be re-elected with a comfortable majority next May, even according to Labour’s own private figures.

These ratings put force behind the question that has hummed around Westminster for three years, spurred by Johnson’s repeated clashes with David Cameron. Will Johnson try to succeed his old rival and fellow Conservative as prime minister?

“Look, what I have said is that I won’t go on [as mayor] after eight years,” Johnson said when pressed. “I think you can go on too long.” What about cutting short the eight years? “No.” So he will serve a full second term? “You betcha!” Suddenly, he sounds more like the Wodehousian figure many adore. Asked whether he could serve as both an MP and mayor, he declined to comment but gave a low laugh.

Many political insiders believe he has a game plan: to stand as an MP, either at the 2015 election or in a by-election after his second mayoral term ends in 2016, and succeed Cameron as Tory leader. That would make him an eventual threat to George Osborne, Cameron’s heir apparent, as well as a continual source of trouble for the prime minister. Prospect has learned that Johnson opposes Cameron’s policy of evicting council estate tenants guilty of participating in August’s riots, questioning how displacing the poor onto the streets fits into a drive to fix “broken Britain.” This is merely the latest in a long list of public and private disagreements with the prime minister.

The Boris dilemma is not just about whether Johnson has reached the limits of his ambition: whether the powers of mayor seem to him like adequate return for such an extravagant endowment of charisma. It is also about whether such a rebellious personality, suited to the big-character canvas of a capital city, disqualifies him from the calculation and compromise of national politics—and whether, if he stays in City Hall, he will try to pull the party (and voters) towards his more liberal view.

***

Some politicians can be identified just by their first name, but not many are instantly recognisable by the back of their head. Johnson’s dishevelled hair, as well as his wit, untidy personal life and ebullient appearances on the TV quiz show Have I Got News for You are part of his charm. “With amazing symmetry, the right seems to have found in Boris Johnson a Tory maverick equivalent of his Labour maverick predecessor as mayor,” says Matthew Parris, his friend and Times columnist. “There’s something about London—indefinable but you can feel, almost touch it—that makes an eccentric individualist with a touch of roguishness, a touch of the joker and a touch of genius, the best and perhaps the only type of candidate that feels right. He doesn’t so much have to do, as to be.”

The roguishness won him early rewards in journalism. He was sacked as the Times Brussels correspondent in the late 1980s for making up a quote, an episode which is now a staple part of his biography. But his enthusiasm for controversy generated hundreds of memorable Telegraph columns, leading to the editorship of the Spectator. His rising profile secured him a seat as Tory MP for Henley and a post as shadow culture minister. But in September 2004, he was prophetic in describing the tensions between his two jobs: “The horses are starting to get further and further apart,” he said, “and the straddling operation is becoming increasingly stressful on the, um, crotch region.”

The following month, the Spectator carried an editorial accusing the people of Liverpool of wallowing in “disproportionate” grief over the killing of Ken Bigley, a Liverpudlian hostage in Iraq. The then Tory leader Michael Howard, already upstaged by Johnson’s easy appeal, ordered his shadow minister to go to Liverpool and say sorry. The offending article was written by Simon Heffer, now a Daily Mail columnist, but Johnson took responsibility as editor. “I will be going… to apologise in person for the offence that I have caused, and to listen in a spirit of complete humility to local people.” Privately, he was said to be reluctant to obey Howard, but he made a concession that few editors would.

It preserved his political career—but only for a few months, when news broke of his affair with Spectator writer Petronella Wyatt, and Michael Howard sacked him from the front bench. Between 2003 and 2005, the antics of the staff at the “Sextator” became a joke; the publisher Kimberley Quinn had an affair with David Blunkett, then home secretary, and with Simon Hoggart, the magazine’s wine reviewer, while the Sunday Times writer Rod Liddle was sleeping with the Spectator’s receptionist. A farce about the magazine, Who’s the Daddy?, opened at the King’s Head Theatre in Islington in 2005. Johnson stepped down as editor that December, the culmination of a bad year.

Yet by May 2008 he had beaten Ken Livingstone to be mayor of London. He had overcome Tory unpopularity in the capital, making inroads into the inner city, as well as the more naturally Conservative “doughnut” of suburbs. Today, despite a mixed record as mayor, he has consolidated this support. As Peter Kellner, president of YouGov, demonstrates in his article on “Boris Labour”, Johnson is far more popular than his party and prime minister. YouGov’s latest figures show Johnson polling at 39 per cent to Livingstone’s 33; nationally, the Tories lag behind Labour on 32 per cent to 51.

That rating partly reflects a sense that he has done well as mayor, despite the limits on what a mayor can actually do. Mayors in Britain do not traditionally have the clout enjoyed by those in France, the United States and elsewhere. “London’s mayor has very few real powers, so it’s inevitable that [mayors] are defined by personality and symbolic policies,” says Kellner. Perhaps for this reason, Livingstone’s tenure was defined by the congestion charge. Will Johnson’s legacy be more than the blue “Boris” bikes for hire which now thread across central London?

The 2007 Local Government Act gave the office new powers in housing and tackling climate change, and more say over planning, waste, health and culture. But as Johnson’s 2008 manifesto put it, transport is “the mayor’s biggest area of responsibility,” and he sets the annual budget for Transport for London (TFL). A common complaint, from Conservatives too, is that Johnson has failed, in the words of one senior councillor, “to tame the big unaccountable beast that is TFL.” One party colleague, a senior Tory MP, says that “transport is key—it has gone backwards under Boris. It is a fucking disaster.” Since Johnson was elected he has raised fares by 44 per cent for a single bus ticket, costing Londoners on average an extra £176 a year, and by 14 per cent for a seven-day zone 1-2 Oyster travel card, costing commuters an extra £240 a year. The latest figures from TFL show that overall delays were up by 16 per cent across the underground network in the year to March 2011.

On traffic, the picture is mixed. A recent TFL report gives some credit to recent initiatives—such as automatic control of traffic signals and better management of road works—for “overall stability in average traffic speeds.” But Dave Hill, the City Hall commentator who monitors London transport, has said Johnson’s 2008 pledges for tackling congestion have yet to be fulfilled, and “the effect of his road management policies as a whole may even turn out to have made the overall situation worse.”

Johnson’s scrapping of the extension to the London congestion charge zone led environmentalists to complain, and TFL predicted a revenue loss of £55bn. But his move was highly popular; he has called it “democratically the right thing to do.”

Elsewhere, he won the battle over Crossrail, defending it and the biggest Tube upgrade in 80 years against the cuts in last year’s government spending review. He succeeded where Livingstone failed in extending Oyster cards to national rail. There are 500 more police on public transport, and his team claims that crime on buses is down 30 per cent. And he has overseen the delivery of the Olympic Games venues on time and under budget.

Overall, Johnson’s performance as mayor has been far better than many predicted. Perhaps this is because, as Andrew Gimson, his biographer, puts it: “Boris is so competitive that he’s determined not to fall flat on his face.” Talking to Prospect, Johnson promised more: “We will be unveiling [before the mayoral elections in May] some radical plans for transport infrastructure and for young people, in the wake of [the looting and riots] we saw in early August, which had many causes.”

In Peter Kellner’s verdict: “Boris has been smart in positioning himself as Mr London versus the national government—on taxes, immigration, housing benefit and so on. He has also, like Ken years ago, established himself as a well-known figure in an age when most cabinet ministers are nonentities as far as the public is concerned.”

***

From this position of strength, Johnson has built a rival pole within his party. It is widely known that there is no love lost between him and Cameron, his fellow former member of the Bullingdon Club as well as old Etonian. “You know Johnson despises Cameron, and says so frequently to those close to him,” says a senior Tory MP. “He says Cameron is intellectually second rate. It goes back to Boris being a scholar at Eton and the Camerons having to pay.” Images of Johnson and Cameron together sum them up: Cameron, the sleek modern politician, neatly turned out, with a fixed grin; Johnson, dishevelled as always but having fun.

They share some political ground: both are in favour of reducing the size of the state, and are countryside-friendly, pro-foxhunting shires Tories. But Johnson’s instinctive libertarianism extends to civil liberties. Like David Davis, he opposed ID cards, opposes extensions to detention without trial, voiced concerns about the targeting of Muslims by police in the wake of September 11th, and railed against “neocons” after the invasion of Iraq. Cameron voted for the war and, in the wake of the August riots, turned to traditional themes of law and order and the importance of family. Johnson, socially liberal, has no time for policies such as tax breaks for married couples. Partly Turkish, he also opposed the cap on immigration first advocated by Howard at the 2005 election and now adopted by Cameron.

“I am a fervent admirer of David Cameron,” he told Prospect. “They [the government] had a difficult hand to play when they got in with the economy and I remain on very good terms with them. But it is my duty to speak up for London, this great city.” Pressed, however, on the visible disagreements with Cameron, he said: “Of course there have been times and will be times when I will be at variance with the government, just as I was with the Labour government, on issues like housing, policing, transport infrastructure, aviation, the burden of taxation and so on.”

He is too coy; on almost every element of that long list, he has not hesitated to challenge the party leader. Now, the news that he  opposes the eviction of council tenants involved in August’s rioting—another key Cameron policy—will dismay No 10, coming just as the party conference is supposed to offer a display of unity. But it may come as no surprise.

In August 2008, just three months after becoming mayor, Johnson described Cameron’s flagship claim that Britain is broken as “piffle.” “If you believe the politicians, we have a broken society, in which the courage and morals of young people have been sapped by welfarism and political correctness. And if you look at what is happening at the Beijing Olympics, you can see what piffle that is,” he wrote in his weekly Telegraph column.

In early 2009, he had a behind-the-scenes row with Cameron and George Osborne, who had vetoed three of his specific proposals for the next general election manifesto: for an estuary airport, more powers for the mayor, and Crossrail. The party conference that year was overshadowed by Johnson calling for a referendum on the European Union’s Lisbon treaty, a move opposed by Cameron, and defending “pariah” bankers from bonus cuts. In July 2010, Johnson took a swipe at Cameron’s Big Society. Arriving for the launch of his cycling “super-highways” that have adorned the city’s roads with blue paint, he said, “We must tackle the scourge of obesity, or the ‘Big Society’ as it’s sometimes known.” It was a joke, but not a respectful one. Just two months later, Johnson provoked serious anger from ministers by comparing the government’s proposed cap to housing benefit to “Kosovo-style social cleansing.”

This year, Johnson has opposed the 50p rate of income tax on those earning over £150,000, and continues to object to the prime minister’s policy of police budget cuts of 20 per cent. “I think that case was always pretty frail, and it has been substantially weakened,” he told the Today programme on 10th August, at the height of the riots—the morning after he had dined with Cameron at No 10. Cameron was said to be furious.

Peter Oborne, the political editor of the Spectator under Johnson and an old friend, believes that he and Cameron tacitly play up the rift. “I believe there is no real contest between Boris Johnson and David Cameron because, whatever way you cut it, Boris is no threat to Cameron. Even if he got into parliament in three years’ time, it would take two years to build up a leadership team. But I think it suits both parties to have a public argument: Boris can define himself by attacking Cameron and Cameron can help define his premiership.” An aide to Livingstone goes further, with a conspiracy theory: “It is quite clear that [Johnson and Cameron] are co-ordinating many of their public rows and they have a common interest in Johnson appearing to be independent.”

***

If he is unlikely to dislodge Cameron, could he eventually succeed him? His style underpins his popularity, yet undermines his image among MPs and in the media as a serious politician and potential prime minister. Even when he toured Clapham Junction in the wake of the riots in the summer, he could not resist a half smile while being rebuked by members of the public, as if he were about to giggle like a schoolboy. The pictures of him brandishing a broom with those volunteering to clean up after the looting must have made many laugh, but perhaps did not entirely reassure them.

The same may be true of his personal life. Last year, fresh reports emerged of strains in Johnson’s marriage, although friends say he and his wife, Marina—with whom he has four children—are determined to make their relationship work. They were on a family camping holiday in Canada when the riots broke out in August. His private life may cost him few votes in London, but might elsewhere. Simon Heffer, a social conservative, wrote in the Telegraph in April 2008: “The guiding theme of his life is the charm of doing nothing properly. His sins themselves are charming in that they are the sort of failings that upset the Edwardians, and few others since. He is pushy, he is thoughtless, he is indiscreet about his private life. None of this matters much to anyone these days, which is why he has gone so far in spite of them, and tomorrow may go further still.”

Asked if he thinks about whether to be more serious, Johnson—for the only time during the interview—sounds as if he does not like the question. “No, no, you can’t worry too much about that. Otherwise you’ll just go mad.” One friend says: “He probably would like to be prime minister, yes. And he may yet be. But he must know that if he’d been more serious, and more careful with his private life, he would stand a greater chance.”

Yet some who know him say that Johnson carefully calculates his “act.” Heffer has written: “I want to dismiss a prejudice about Mr Johnson, and I do so as one who has known him for the past 20 years. It is that he is a buffoon. He isn’t. The act is calculated and it has required serious application and timing of the sort of which only a clever man is capable.”

A parallel criticism of Johnson, friends and enemies agree, is his disregard for detail. One Tory MP recalls the resentment colleagues felt over Johnson “not pulling his weight” when shadow minister. A senior figure in the City concurs that “Boris is very charming to work with but you can tell he gets bored the second discussions focus in on detail. Now, this may be no bad thing in a prime minister or even a mayor—you can delegate a lot. But it can be disconcerting.”

***

His personality could be one obstacle; another is the nature of the mayor’s role. “Boris is the perfect mayor in that he cuts across the political divide. He is the Heineken politician, reaching parts of the country other Tories cannot reach,” says one senior Conservative. “As in every city in the world, a mayoral candidate needs to be a big personality who appeals beyond his or her own party.” But the more he distances himself from Cameron, the more he angers the MPs who would one day have to elect him leader. Tory leadership duels are only fought among all the party members after the MPs have eliminated all but two in a series of secret ballots.

“The trouble for Boris is that being leader requires appealing to the party and not the country,” says a Tory MP. “Look at Ken Clarke. The country loved him but the party did not… To get elected party leader you have to win the hearts and minds of some very dull people, and it has never struck me that Boris Johnson has much time for doing that.”

Others warn that his decisions as mayor may serve as a hindrance later. His main achievement, says Ben Rogers, director of the Centre for London think-tank, has been “to secure a very good settlement for London in the last spending review, especially for transport. That should give him a good platform on which to campaign for London.” He adds: “As for any other ambitions he might have, electoral success in London is no base for success nationally. The London electorate is so different from the national one—his support for immigration and bankers for example, probably goes down better in London than it would across the country.”

Peter Kellner observes: “Were this America, you could see Boris running for national leadership from the mayor’s office. But as ours is a parliamentary system, prime ministers and party leaders have emerged from the Commons ever since 1906, apart from Lord Home in 1963.”

***

If Johnson wins a second term, as expected, he will doubtless continue to be a thorn in the side of Cameron and the government. The Tory leadership could make life difficult for him in return. “Boris will continue to be a pain, but the response will get harsher, especially if he wins next year,” says one Tory MP. “That will panic high command because we will be at a low point nationally. You will see Osborne cutting down his funds. This is what chancellors do to cut off their rivals: Brown took them down one by one.”

Indeed, as Peter Oborne puts it, “If there is a rivalry, it is between Boris and Osborne.” Should Johnson decide to bid for Downing Street after his second term, he is more likely to come up against the chancellor. Osborne is believed to have disapproved of Johnson being the Tories’ mayoral candidate in 2008, and ordered Conservative Central Office to keep him under control during the campaign. Now he holds the purse-strings, Osborne has done his best to fend off, albeit unsuccessfully, some of Johnson’s most keenly pursued projects, like Crossrail.

The outcome of this rivalry will depend a lot on the economy. As a Treasury official points out, “If George can turn things round by 2015, then many MPs elected or re-elected in that election will owe their seats to Osborne—not Boris.” But others think Johnson stands a better chance of succeeding Cameron: “Johnson has ‘cut through’ in a way George can only fantasise about,” says one new MP. “The fact that many of us are new means we don’t know him other than what we see on TV, so the jury is out.”

In the end, there is little that unites a political party more than success. Gimson believes that Tory MPs will “swing behind” Johnson if they feel he is a winner—which he will be if he achieves a solid second victory, against the handicaps of London’s propensity to support Labour, and anger at the Tory-led government’s cuts. However, even if the polls are reversed and he loses to Livingstone next May, Johnson’s political career may not be over. He could easily return to punditry: it has been widely reported that he commands £250,000 a year from his Telegraph column (a sum he has described as “chicken feed”). But some say that, even in defeat, he would be well-placed for a Commons return. “There will always be a safe seat on hold for someone like Boris,” says an experienced Conservative MP.

Johnson’s behaviour shows that he is serious about securing another term as mayor. His positions on immigration, marriage and much else may be too liberal for the party faithful. Yet he may well find a way to go further. “There have always been others to worry about the practicalities, just as there are to do the laundry,” says Gimson. “When it comes to his plans, Boris has always been very disinclined to allow mere practical questions to get in the way.” If he can get the balance right, between dissent, humour and serious policies, he could go all the way.

He says:

“[I am] a wise guy playing the fool to win.” Sunday Times, July 2000

“Voting Tory will cause your wife to have bigger breasts and increase your chances of owning a BMW M3.” Said during 2005 general election

“My friends, as I have discovered myself, there are no disasters, only opportunities. And, indeed, opportunities for fresh disasters.” On being sacked from the shadow cabinet, Telegraph column, December 2004

“Yes, cannabis is dangerous, but no more than other perfectly legal drugs. It’s time for a rethink, and the Tory party—the funkiest, most jiving party on Earth—is where it’s happening.” Telegraph, July 2001

“I have as much chance of becoming prime minister as of being decapitated by a Frisbee or of finding Elvis.” Daily Mail, July 2003

They say:

“I don’t always agree with him but I respect the fact that he’s absolutely his own man.” David Cameron, March 2008

“The most ambitious person I’ve ever met.” Simon Heffer, Telegraph, April 2008

“You are a self-centred, pompous twit. Even your body language on TV is pathetic. Get out of public life. Go and do something in the private sector.” Paul Bigley (brother of murdered hostage Kenneth Bigley) to Johnson on Radio City in Liverpool, 21st October 2004

“What the f*** are you doing here?” A drug dealer, as Boris burst into the man’s house as part of a police raid.

The last two quotes are taken from The Bigger Book of Boris (Biteback)

“Boris Labour” – Ken Livingstone’s biggest problem is the one in five Labour voters who prefer Boris, says Peter Kellner