The Uses of Pessimism and the Danger of False Hope by Roger Scruton (Atlantic, £15.99)

The Rational Optimist: How Prosperity Evolves by Matt Ridley (4th Estate, £20)

The contrast between the titles of these books is a little misleading, since they have a great deal in common. Each of them could have been read with pleasure by Margaret Thatcher, whom Scruton admired, and in whose government Matt Ridley’s uncle—Nicholas Ridley, the minister of state responsible for the Falkland Islands—played such a prominent role. Scruton’s book, as befits that of a philosopher, is more nuanced, more profound, and is in many ways more pleasing than Ridley’s, whose one-dimensional message becomes a little tiresome. But both are brimful of intelligence and learning.

For Scruton, the uses of pessimism are to counter various fallacies that underpin many utopian and visionary political philosophies. He elegantly skewers the mistakes involved in myopically concentrating on the best-case scenario, as gamblers do when they don’t admit they are taking risks. He magnificently demolishes Rousseau, whose belief that before civil society men were “born free” underpins so much utopian thinking, and its associated habit of finding sacrificial victims such as kulaks, Jews, or yesterday’s comrades in the revolution when utopia fails to materialise.

He points out the error in supposing that all human transactions are zero-sum, so that if A is rich and B poor, one’s success can automatically be assumed to be the mirror image of the other’s failure, giving B just cause to resent A. He loathes “top-down” planning—as does Ridley—but Scruton’s venom against the planners in Brussels is incomparable. He excoriates the “aggregation” fallacy which supposes that you can simply heap good things together (liberty, equality, fraternity) without trade-offs, refusing to accept that different types of good can diminish or even cancel out each other. On the way, Scruton scatters valuable insights into terrorism and its mindset, the Islamist opposition to usury, the errors of modern architecture, the idiocies of farming policy, and much else.

Once a small dose of pessimism douses the flames of conviction, what remains? Scruton’s spiritual ancestors are Burke, Hayek, and Oakeshott, and the objects of his admiration are the long, faltering, experimental, and eventually time-honoured institutions and habits that have led to civil society—to the replacement of “I” by “we.” One of his paradigms is the common law, with its accumulated record of ever-increasing wisdom: the practical equivalent of science’s long history of conjectures and refutations. Another is the old European or Levantine city, with its unplanned, yet appropriate alleys and neighbourhoods, where people have lived and worked for centuries. The old city serves as an emblem of rooted and cultured togetherness; new cities by contrast tend to be huge and brutal affronts to anything resembling a decent life. There is an element of sentimentality here—old cities were frequently scenes of massacre and pogrom—but also an element of truth.

Turning to Matt Ridley, I suppose it is no surprise that he is an optimist, given the events that took place during his tenure as non-executive chairman of Northern Rock between 2004 and 2007. Ridley does not see it like that—since apparently “it was [the] government’s housing and monetary policy” that was to blame for the company’s crisis. Anyhow, busts are just inevitable bad patches imperceptibly slowing the booming juggernaut of economic progress.

This may just be a beautiful example of Scruton’s diagnosis of the gambling mentality, in which failure, when admitted, is due to malign fate rather than personal error. If we put it to one side, what remains is a forceful, rhetorically brilliant, but eventually repetitive hymn to human progress, the key to which is trade, which brings innovation and prosperity. The book is dense with figures telling us how trade has made us better off. Now we need only work for milliseconds to purchase things that would have cost our ancestors hours of labour; the supermarkets provide us with a better range of choice than Louis XIV’s 498 kitchen servants could provide for him; again and again the fears of pessimists have proved unjustified. Everything progresses thanks to human inventiveness, but above all thanks to trade and capital. The poor are swept upwards, our moral sensibilities improve, and happiness increases alongside wealth. Governments and churches are but obstacles to this upward sweep, which is set to continue into the future as new sources of energy—not the green nonsense that needs hefty government subsidies, but the brainchildren of bold entrepreneurs—ensure a soft landing for the carbon economy.

The Adam Smith who wrote, “People of the same trade seldom meet together even for merriment and diversion, but the conversation ends in a conspiracy against the public or some contrivance to raise prices” is evidently not invited to this feast, although the invisible hand certainly is. There are grudging mentions of monopoly, theft, fraud and other little bits of sand in the machinery, but we can be sure they require only a tiny bit of government attention, although Ridley claims he always supported government regulation of markets in capital and assets. Trade unions, more or less identified with craft guilds, are given four mentions, all negative, although why concerted action to raise the price of labour escapes the showers of applause accorded to other practices, such as concerted action to depress it, is not explained. Health and safety in the workplace are not among the cherries that Ridley chooses to pick.

It is difficult to resist this torrent of belief, but sometimes we must. In the prologue, for example, we are told that experiments by economist Vernon Smith and colleagues confirmed that “markets for goods and services—haircuts and hamburgers—work so well that it is hard to design them so they fail to deliver efficiency and innovation…” Haircuts? Hamburgers? The market in one is largely an exercise in stoking up and then satisfying neurotic vanity, and the market in the other is largely one in which poor quality meat is passed off onto an obese and self-destructive public.

Perhaps the difference of emphasis in the two books is best seen if we reflect that while Scruton is conservative in the sense of decently respectful of tradition, the market is not. If a 14th-century church can be bought, demolished and turned into a Tesco or a Wal-Mart (a company about which Ridley waxes especially lyrical) then it will be. Consumers will be better off. Many of them will not even notice that they have lost anything. Spiritual capital has no market value, unless it can be translated into the heritage industry. Which may remind us that knowledge is not exempt from market forces. Airbrushed, distorted and Disneyfied, fiction often pays better than sober fact.

But isn’t the market irresistible? Saying so would expose the fundamentalist to another of Scruton’s charges, that of the “moving spirit fallacy,” in which we suppose it is futile to oppose whatever is the spirit of the age. Of course, it often is. But that is why, over time, things like planning controls—nasty governmental restrictions on the right of capital to do what it likes—have proved their worth.

We can listen to both Scruton and Ridley, and it is not a case of glass half-empty versus glass half-full. We can aspire to the best of both. If we are pious towards culture, knowledge, tradition, co-operation and community we will take more from the first, yet Ridley is right to praise the present compared to the past, by almost all the indicators that are easy to connect to human happiness, except perhaps the virtues, including modesty and stoicism, which have seemed important to other writers. He may even be right to envision a glorious future. He quotes Macaulay, whose Whig view of history he shares, asking why “when we see nothing but improvement behind us, we are to expect nothing but deterioration before us?”

Unfortunately, the answer is that progress is one uniformity among others. The world is littered with deserted cities, and empires have one after the other decayed and gone. We should do better cautiously to ponder Russell’s remark that “the man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken.”

Scepticism, which in this context may seem pessimistic, is never a bad thing.

4084069135cc286edf14d23.58439815