Log In | Subscribe

The end of the journalists’ monopoly

Tom Streithorst
YouTube: everyone can be a journalist

With YouTube, everyone can be a director now

“It’s 2010—time to get off of the Titanic and get onto the lifeboats,” Kevin Anderson, the Guardian’s digital research editor, told me. I wasn’t pleased. I thought: ”But the Titanic has an orchestra, a well-stocked bar, fine Egyptian cotton sheets. The lifeboats have nothing but freezing cold water.” I was at an event at the Frontline Club which sought to ask how the news industry will survive the next decade. Will the internet save it, or kill it? I came out convinced the internet will be good for journalism, but probably be bad for me, a journalist.

For news consumers, the internet rocks. Wikipedia allows us to find out everything about just about anything—no need to take the bus to the British Library. The best writing about the financial crisis hasn’t been from traditional journalists but rather in blogs from academics like Simon Johnson or traders like Barry Ritholtz. Riverbendblog may have been opinionated and occasionally inaccurate, but it gave a better picture of life in Baghdad in the dark days of 2005-06 than any western journalist could. Ten years ago we would not have heard these voices. Now they are omnipresent.

Read more »

Prospect talks to the father of the world wide web

Tom Chatfield

Above: Tim Berners-Lee at TED2009—demanding “raw data now!”

On 5th January, just before the public launch of data.gov.uk, Prospect’s Tom Chatfield spoke to Tim Berners-Lee about how Berners-Lee helped the government to open up public data. On 7th January, Chatfield spoke to Berners-Lee’s friend and colleague Nigel Shadbolt about his role in the project. The edited highlights of both conversations are below.

The full inside story of the government data initiative is told in the February issue of Prospect, and can be read online by subscribers here. A preview of the article, by James Crabtree, can be read for free on our website here.


Tim Berners-Lee in conversation

Tim Berners-Lee studied physics at Oxford, before working in telecoms and software engineering. In 1989, while working as a fellow at the CERN research centre in Switzerland, he published the academic paper that defined what would come to be known as the world wide web. In 1994, he founded the World Wide Web Consortium, a group devoted to keeping the seething mass of pages he helped to create working together. In June 2009, alongside Nigel Shadbolt, he was appointed as an information advisor to the British government.

Tom Chatfield: How did you come to be working with the British government?

Tim Berners-Lee: It began with a lunch at Chequers, when the prime minister asked me what I felt the UK should do in order to make the best use of the internet, and I said, you should put all your government data onto the web. And he said, okay then, let’s do it. [laughs] So when one has spent a lot of one’s life persuading people to put things onto the web, and persuading people to be open, it’s almost disarming to have somebody say that straight away. The result of that was a team in the Cabinet office of a team under Andrew Stott. Various people in the UK government had experience of this area already, so it was a question of how to accelerate this as much as possible.

TC: I know it’s sometimes felt in the world of online innovation that governments are not the best people for creating radical change. What motivated you to put this effort into working with and within the British government?

TBL: The neat thing about this is that there is such a clear win: the data we’re talking about has already seen a lot of effort expended on creating it. It’s a valuable resource that has been produced by parliament for a particular purpose, and it has typically been sitting there as a really valuable but under-used resource.

The thing people were amazed about with the web itself is that when you put something online, you don’t know who is going to use it. You’re looking for something and you think it’s impossible that somebody will have done it before, but you find that they have, and the web saves your bacon. It’s the serendipity—the unexpected reuse that is the value of the web. When you move to data, suddenly this is not applied because data actually is really desperately boring when you look at it by itself. When you put data together you can derive very powerful new insights; so I think that realisation that the UK had got all all this resource that was under-utilized means the arguments become very obvious for putting them out there for people to re-use.

TC: You’ve been working in this field for some considerable time, but is this the first time you have worked in this way with a government?

TBL: I have encouraged governments on various occasions to adopt standards and to use them for websites, but this is the first time within government. What happened is that at the beginning of 2009 I decided that this is going to be the year in which I ask people to put data on the web. I gave a talk at TED, including getting people to chant “get raw data now.”

TC: Part of the interest of the story seems to be that from this first encounter with the prime minister there was a degree of serendipity involved: the time proved incredibly ripe in government for this initiative to gather a huge amount of momentum over a very short time in government terms.

TBL: People had different points of view coming from different places, but the consistency of the encouragement that we’ve had has been very gratifying. And I suppose this is really my first experience with getting involved with policy so I don’t have very much to compare it with, which means that my impression is that if people really do want to do something, and they are excited about it, then they can. I think we have to be very careful about having a burst of apparent momentum, though, and a lot of talk; we have to realise that it is going to mean a lot of pushing of other people, staying late for a bit, putting extra effort in maybe to go to a seminar to learn how to do things with linked data. I think that within departments there is bound to be resistance from a few individuals, who are going to have to be coaxed around it.

One of the really important things of course is that we should do all this without changing the way people work. It needs a very concerted ongoing push at each level: managerial, within departments, the material level, and the grass roots. This stuff happens because people are dedicated and put a lot of time into doing it. And then it’s great to celebrate the results: things like these “hackathons” which we had a couple of, where people got together and made visualisations of the existing data. Those have been good because they give you a good sense of the data being made into something really interesting.

TC: Did you feel that people really grasped what you were talking about, intellectually, the principles and ideas?

TBL: Yes. Obviously from the point of view of the timing it connected to a concern about transparency in government, which is not just the UK, the US is also very concerned at the moment; some people connected it directly to that, and that may have helped, the sense that this was holding the government accountable. One of my fears is that people would see that as the only motivator and they wouldn’t see that we also have an enormously valuable resource.

For instance, somebody blogged on DirectGov, putting up some data which was just the grid references and years of bike accidents over three years. And that was on 10th March, I think, and later on that day somebody pointed out that it had been put up in Microsoft Excel, and said you shouldn’t do that, it’s a proprietary format, you should put it up as a comma separated file which anybody can read, and by the way here it is turned into a csv; and then someone says they’ve turned it into a kml file which can be used with a mapping application; and then the next day someone from the Times blog says I have done the mash-up, so here is a map you can go to and zoom all over the location and find your journey to work and see where all the bike accidents have been and maybe modify your journey to take another route. That was within 48 hours: the data had been turned from a pile of figures into a really valuable resource, which can save lives, which perhaps can help in the long term helping the public put pressure on the government to deal with black spots, and that is immediately useful to anyone getting on a bike.

Now imagine if a government department in any country had decided they were going to have a bicycle accident website. They would probably have spent a long time drawing up a requirements document, put it out to tender, and eventually gone for the lowest bidder, and after a certain amount of time the company would have come in, and then there would have been a review, and eventually the site would have been launched, and with luck it would have been useful; but in fact the message is that there are people out there who are prepared to put the effort in to turn data around before you have gone to the trouble of doing it yourself. It’s about seeing whether the mash-up-sphere, if you like, will do it for you. And that sphere will always win because they have access to data from different departments and non-government sites and all kinds of things. Somebody who is out there mashing up data sources, or someone in government doing that, is always going to produce things that go far beyond one single data set.

TC: Coming in from the outside, how did you find the internal working practices of government?

TBL: The people I met were generally very switched on, and I have been very impressed with the way that people in the Cabinet Office have made things happen and explained to me how things work. Yes, people tend to send around word processor files in emails, where at W3C everything is on the web. The British library has I think one of the largest public wi-fi areas in the country, possibly in the world, but the government doesn’t have open wi-fi so one has to work around that, you can’t just open your laptop and be connected. But I wasn’t there to complain or worry about that, and of course there are an awful lot of industries in the world that still operate by sending around copies of a document via email.

TC: One of the key things seems to be the Ordinance Survey data. As I understand it, you went in thinking that OS would not be your prime focus, but it ended up becoming a key component. Was there a eureka moment with OS data?

TBL: My initial feeling was that OS had a complex history, that the whole set-up for OS as a trading organisation was defined, so that it was not something to deal with in the first instance. But so many people we came to, who deal with data of almost any variety, said that government is to do with the government of the country, the place, and almost everything you do is to do with some physical place on a map. We met so many people who were very constrained by OS data or had a governmental right to use the data but as a result couldn’t pass on the information they created to the public, so we were under a huge amount of pressure from a large number of people to do something.

TC: I know historically it has proved very difficult to open up the OS data. Did you find there was a lot of opposition to that?

TBL: There had been various attempts to review the problem, some of which had been very conservative and focussed on small changes to the model, but there was one report that focused on the economic side and said very strongly that everything should just be made public.

TC: Rufus Pollock’s report?

TBL: That sounds like it. Basically, the ideas in the report are correct. But one of the problems is that the value to the individual citizen of having the data available, that return on investment, is very difficult to measure. How can you measure the value in your life of the web: how do you put a pound sticker on it? You can’t. You can try in some ways, you can say you would have wasted this much time going down to the library which I don’t now do, but the whole thing is wrapped up and you do things now that you didn’t used to do. But this is one of the problems.

One of the things that was very important to us was to preserve the OS. A lot of us remembered at school being taught to use an OS map: we grew up on our holidays using OS maps to avoid traffic jams, find beaches, walk through the hills; and there are a huge number of people in Britain who are very attached to the Ordinance Survey and who value it, they know the OS as the people who make their maps, and are a jewel in the crown of Britain’s information resources. A lot of it is sentimental attachment too, to particular maps and the particular way they are presented.

TC: What are you worried about? Are there some great obstacles remaining—and is it possible that a different government might not care for the agenda so much?

TBL: The whole openness of data thing is so non-partisan that I can’t see a different government really wanting to reel back the openness. I think that once people have seen it, too, it will be easier to see that nothing horrible has happened. The fears that people tend to have when they are managing a particular bit of data sitting at a computer are to say, well, I’m worried that people will misinterpret the data, I’m worried that people will use it for the wrong thing, or I’m worried that people will think it’s more accurate than it is. Those are the sorts of things that you hear, from the very large standard excuse set. But once the data is out there those sort of excuses won’t be there any more, because people will say, well, the data is out there and it’s not really being abused, and people do understand that it’s not very clean data and that nobody is perfect, but they are very grateful to you for making it available.

The things that I am concerned about: we need to keep the momentum going, and ensure that people are following up with data sets. There is also a temptation to mail out a DVD and say, okay, here’s the data, but obviously the data is changing and being made open is just part of the cycle of the development of data; we have to grow to learn how best to do this.

TC: Is this an area that Britain could lead the world in, or are there other countries we should be looking to and trying to emulate?

TBL: America is also at it, of course, with its data.gov site.

TC: They seem to have less emphasis on making it highly usable for developers and third-party APIs.

TBL: That’s right. I think when it comes to the quality of data and making it usable, Britain is ahead. Of course it is early days. There is an awful lot of data out there: in both countries there has been a call for a list of the things that are out there, and I think just producing that list of what data to call for is quite difficult. We need to move to an ethos where if somebody in government creates a database then by default they will create a path to making that available and usable publicly. There are different ways in which the UK and US can learn from each other. There are also efforts go on in other countries—Australia, New Zealand, Toronto, New York, the State of Massachusetts, all have public data projects.

TC: Is this a movement that could change the way people think about politics and interact with political systems?

TBL: Yes, I think it will have a big effect: the accountability of government and transparency will have a very healthy effect on the way that government is run.

I felt initially that we clearly we needed to do this with the most developed countries, who understand about putting stuff on the web. But people are also pushing the idea of this in developing countries, because that’s where government and data transparency is needed, and you really need to establish trust in the government in order to justify investment from outside for example. When I was recently in Uganda, talking to ministers and the prime minister there, I took the opportunity to mention the openness of data in Uganda, so it may be that some of the most important effects that you find early on actually come from developing countries.

TC: Do you see yourself having a long-term involvement with the government, or with governments, on this?

TBL: This has been a project of a certain length of time. I hope that the momentum that it has got will be self-driving, I hope this will take off exponentially, and that I will be able in future years be able to push other sorts of things. What should it be in 2010—should it be the year of scientific data, social networking data? There’s a lot of ways in which we have to go in how we use the web, and they all connect together. But putting government data on the web has been a very exciting journey. We have to keep pushing, though. Constant vigilance.

TC: And what about your personal motivation. Obviously you’re very driven: it would be quite possible for you to sit back if you wanted to.

TBL: It is very exciting, clearly, to make things that work and that allow computers to do things that help us. The whole new field of web science, learning about how the web as a very large system and how humanity connected by technology should evolve, has a lot of excitement. But there is also a certain amount of duty. The web is this big system which we did actually make, this artificial system created by the people  who sit down and write protocol and machine specifications. The way computers interact over the web is defined by a system we invented and can change. We have a duty to make sure that at the same time as we are putting data on the web, that we look more broadly to make sure the web does serve humanity, and think about the 75 to 80 per cent of the world who don’t use it at all at the moment. As always, too, I’m motivated by meeting people who are also very fired up: people like Andrew Stott, who are excited about doing something that is going to be good and effective.

TC: What most concerns you, and most excites you, about the future of the web?

TBL: Most of my concerns are to do with the web being controlled by one party or one group, whether government or a large company that has got excessively powerful and is able to control what one sees or know what one does; if a government decides it is going to control or limit what people do, or spy on them. Those are the main fears.

Nigel Shadbolt in conversation

Nigel Shadbolt is Professor of Artificial Intelligence (AI) and Deputy Head (Research) of the School of Electronics and Computer Science at the University of Southampton. He is a Director of the Web Science Trust, and of the Web Foundation—organisations committed to advancing our understanding of the web and promoting the web’s positive impact on society.

Tom Chatfield: Where did this begin for you?

Nigel Shadbolt: I moved down to Southampton University in 2000. The whole area of the semantic web seemed to me really exciting and I was fortunate enough to lead a project started in 2001 called Advanced Knowledge Technologies or the AKT project. It involved 5 universities, really looking to try and build technology and methods in anticipation of more and more information management happening on the web. As you’ll probably be aware, Tim has an appointment at Southampton. I was talking to him at a web conference in New York in the mid-2000s and he was saying, look I really want to try and get the semantic web, the ideas of linking data to the web, firmly established in a European context and you guys at Southampton with this AKT work are really showing what the art of the possible is. That’s really where our contact, our personal collaboration, started.

What really happened then was one piece of serendipity. John Sheridan, who was at that time working for Carol Tullo at the Office for Public Sector Information, had approached me and he had been terribly interested in the idea of using these techniques for the linking of and the publication of public sector information. This was back in 2004—and we decided halfway through the AKT project that public sector information could be a really interesting test case for us because it was a perfect experimental environment for exploring our techniques.

Everybody knows about data protection, but few people know that there is also a high level, legally binding directive to make public sector information available freely across Europe. It seemed like a good context, so we launched a pilot project called Active PSI, and that was taken up by the OPSI people and they started to push it around departments, local authorities and so by 2005 or so, 2006 maybe, these experiments were being reported into parliament in the Ministry of Justice documents talking about possible use of this technology.

So we had a community within the civil service who were exploring these ideas and seeing utility—and then we had Tim, who could go out and command huge respect to evangelise this new approach for semantic technologies.

TC: His is a name that can make things happen.

NS: Yes! And it really does make a huge difference. If you remember when the first web really took off, the first place it got wider adoption was amongst scientists and CERN who were using it to share information. That’s why Tim built the first web, because they had so many different documents on so many different systems, it was just a pain to constantly have to move between them. So they built this layer that would sit across them. That was, if you like, an incubator group for this version of the web. And I’m starting to believe that public sector information could be a similar incubator for this new data web that we’re seeing emerge.

TC: And you and Tim were appointed as government advisors in June 2009.

NS: Yes. It’s an interesting category, “advisor.” What we did with our terms of reference was two things that were interesting. One, make them very specific about deliverables. This would of course have to ultimately involve policy and hard policy decisions—but it was also about a single point of access where the information assets would be catalogued and described—that is the concept of data.gov.uk. Also, the other thing was that we were going to build this thing using what people referred to as “agile” programming or project management, where we really are building small but fast.

TC: So you’re bringing in best practice from the IT and academic industries?

NS: Yes, trying to do stuff fast but sustainably, using open source where we can.

TC: I presume you’re quite adept at avoiding what we might call the public sector traps of taking a very long time to get terms or batting around committees before you begin to get progress…

NS: Yes. And that is why, so far, I think things have really moved quite quickly.

One of the really fascinating things about this work is that our claim is that you can never anticipate the services that will actually turn out to be the killer apps. You always recognise them after the fact, but never understand them before, and that is precisely because human ingenuity always outruns your idea that you should give people this service or that service. We must try to turn the service provision mentality on its head: provide the basic raw material for services and the services are the creatures of people’s commercial and social and voluntary efforts.

TC: If you want to offer people something transformative you cannot possibly plan for that: you just create the conditions in which the unexpected is able to arrive?

NS: I actually think that’s the key to a lot of innovative processes. There’s a continuum of usability here which is very hard to appreciate: when you are making access to data restrictive or expensive you don’t see the long tail of applications that will naturally arise on the web. Lots of applications will be used by a few people but because it’s easy to build those things and free and unrestricted to do so it’s worth people putting the effort in.

TC: I know the Cabinet Office people are very much up to speed, but are there notable data-huggers around or potential sticking points?

NS: There will be other key data sets for us that we’ll have to get resolved and some of these are historic. This gets us onto policy, which I think is interesting. We can do the technical piece but part of the terms of reference was to look at policy and regulatory process. One thing that’s happened is that when the rail and transport franchises were let, they let the information go with the franchise at the level of something like your railway timetable. What it means is that the train operators have built a little iPhone app to tell you about train delivery and timetables and if anybody else tries to take timetable data and publish an app they are told to cease and desist.

Information about where the bus stops are, when the buses run, when the trains run—we would assert if they are paid for by the taxpayer, this is public information. And there the question is that it’s not that the operator is doing anything particularly wrong, it’s just that the information went with the franchise. As these things get renegotiated or as we look forward we have to think about unblocking and keeping information available for unrestrictive use in these contexts. We’re serious about this and certain classes of information need to be in that space. So sometimes policy has to think a little bit about where the information should or shouldn’t reside.

TC: Will everything on the data.gov.uk site be up to the linked data standards?

NS: Not everything in the first instance, because there is a lot and we can’t convert all of that immediately. But we’ve got lots of willing hands who are willing to do that and there will be a very interesting role for crowd-sourcing in this area.

TC: And what is working with Tim like, as a colleague?

NS: Well, we’re friends as well as colleagues so it’s great to have a project that we both feel so passionately about absorbing our interest. You asked, where is Tim’s commitment to this—I think in this area the thing to say is that he’s extremely proud of being a British subject, and who feels pretty strongly about this country and this country being a good place to work and be in and support. And it so happens that this is a really good crucible for a large part of what he believes we need to do for the evolution of the web.

Also, I’ve been working with Tim on our other pet project, to try and get the discipline and study of this thing we call “web science” off the ground. Web science is really the view that says, look, the public sector project is one manifestation of a huge and much bigger challenge, which is that the web is now a complete ecosystem that’s changed the world and we don’t really study it as a complete system, and we need to think about doing that.

TC: And we need a rational, analytical basis for doing that rather than just vague speculations

NS: It’s not just engineering, it absolutely does require input from sociologists, economists, lawyers and such-like. The web is humanity connected because it’s people and advanced information processing systems; you’ve got this really interesting ecology evolving. Can we design for it, can we anticipate it, can we understand it?

Tim and I wrote a Scientific American article together about a year ago that’s got a nice summary of what this web science is and refers to the semantic web. What I would say is that working with Tim is a delight. He enables, he gets listened to, and of course opens doors. That’s the first and obvious point. But he is also passionate about it, he cares about it, he is technically extremely and extraordinarily close to developments—so he really does care about standards, it isn’t that he just sits at the top. And working with him is very much like working with an equal, he really does understand that in terms of trying to make this work in a UK context.

TC: And I think that many people in a similar position to him might feel that the cutting edge was a long way away from governement.

NS: My group did the first linked data pilot with OPSI in about 2004 and over the next couple of years he kept hearing me say this public sector information is much more interesting than I expected, and he would say yes it’s interesting, because it has so many ingredients that really work as the data web incubator. I think that’s the opportunity we both saw, that actually, surprisingly, this is a place to push and with everything around transparency, accountability, that’s now in the public context as well…

TC: …and it’s in this area you can deliver the decisive proof of the power of these ideas. I was absolutely struck after seeing him at TED, and having spoken to him, by just how driven he was; considering he could spend the rest of his life collecting honorary degrees. He seems to be extraordinarily disciplined, by any standards, about how he spends and allocates his time and his energies.

NS: I think the real thing is that he’s absolutely ruthless about that—in terms of mere journalistic vanity or air-time on the TV, that just doesn’t count. And in fact he’s notoriously hard to get an interview or a quote with.

TC: And how does he tolerate fools? I guess there must be frustrations.

NS: I think it’s very striking actually that he will often devote a large amount of time to what seems to be at first quite a simple-minded question but actually is touching on something pretty basic. Tim has been around, pushing the web before the web, and as it really took off has heard so many of the standard misconceptions or worries or genuine concerns that he has acquired a very deep feel for where issues can arise.

I think, when it comes to obdurate stupidity, there he’s got a pretty good line in saying “you know, you’re just wrong… you’ve just got your nose in the wrong place!”

TC: What about the future of the world-wide web itself as more people around the world come online?

NS: There are some critical properties of the web which we want to ensure and endeavour to protect. We talk about universality of access and non-proprietorial standards, these kind of issues. There are always forces who would love to lock information up behind walls, because that gives them a natural monopoly and they can charge a lot of money for it, or for various reasons of control or power. And then there’s the dark side of the web, the way in which it can be subverted for crime or a host of other darker things; but that’s humanity. In a sense, the bit that we have to argue for and push for and be on our guard for is the fundamental building blocks: are the standards open, are they genuinely accessible by all.

The majority of the world is still not connected, too, and there’s a huge issue there about disenfranchisement for those who aren’t. So it all adds up to me again to an issue for us in web science, which is, could it [the Web] fragment? There’s no natural reason why the system should endure and persist in the way it is at the moment. But you really would be alarmed if we were back in the bad old days of serious information disaggregation.

Mash the state

Tom Chatfield

It all began with a lunch. Tim Berners-Lee, the father of the world wide web, was invited to Chequers in spring 2009. A government taskforce had just published a report aimed at making Britain a digital world leader and technological reform was in the air. Even so, Berners-Lee was surprised at what came next. “The prime minister asked me what Britain should do in order to make the best use of the internet,” he told Prospect in early January. “I said, you should put all your government data onto the web. And he said, let’s do it.” A month later, Berners-Lee flew in from his base at MIT in Boston for a meeting, this time a cup of tea with Brown in the garden at No 10. He brought with him his friend and colleague Nigel Shadbolt, a professor of artificial intelligence at Southampton University, who works on next generation web technology and has piloted his work on public data. Sitting in wicker chairs, they hatched a plan for a new government team, led by Berners-Lee, to unlock Britain’s public data.

On 21st January this year, less than 12 months later, the government launched a website to do just that (you may have seen the television adverts). Modelled on a similar effort by President Obama, data.gov.uk brings together over 2,500 public data sets, ranging from abandoned vehicles and A&E stats to child tax credits and carbon indicators. And Brown has promised, in a few months’ time, to open up the jewel in Britain’s data crown: the maps made by Ordnance Survey.

“It can be tricky to explain why Tim’s work matters so much,” says dotcom entrepreneur turned government adviser Martha Lane Fox. “But the data he has been able to release can reorder the balance of power between the citizen and the state.” Such claims are often made for “e-government,” whose hype is traditionally exceeded only by the price tags attached to the (often disastrous) IT projects undertaken in its name. But Berners-Lee’s work has the potential to be different, relying as it does on an unprecedented combination of technology experts, amateurs and businesses like Dr Foster or Experian to take the new information and present it usefully. This helps people make better decisions, underpins information-age businesses, and—because it to some degree redraws the boundary between people and government—may also change the terms of politics. Yet perhaps the most remarkable fact is that it happened at all. Others have tried to unlock Britain’s data, only to run into walls of official obstinacy, vested interest and political indifference. So how did Berners-Lee do it?

Read more »

Whitehall’s web revolution: the inside story

James Crabtree

Above: Tim Berners-Lee, the inventor of the world wide web


This story is now line for subscribers to read, while an exclusive interview with Tim Berners-Lee and Nigel Shadbolt is available to read free, here.

Before working as an editor at Prospect, I was briefly a civil servant. The experience taught me that most civil servants knew nothing about data, and the few that did were rarely listened to. Most were masters at prevarication when anyone tried to suggest that they open up vital information about schools, housing, health services, to the public; the crown jewel in Britain’s data crown, Ordnance Survey, was especially jealously guarded. So I was a surprised—shocked, even—to learn just before Christmas that the deal was done. An infrastructure for the mass release of data into the public domain was in place: in a few months they would be giving it away for free. How had this minor policy miracle happened? Three words, I was told: Tim Berners-Lee. It seemed the inventor of the world wide web, and one of Gordon Brown’s boldest and unlikeliest appointments of the last year, had winkled open the treasure chest.

For the last six months he and his friend Nigel Shadbolt have been leading an unlikely, quiet crusade inside Whitehall. This morning both both Berners-Lee and I discussed the implications of what they have been up to on the Today programme.

Some of Britain’s most impressive internet policy experts had long been trying to break down this particular door. Ex-MP Richard Allan. Cabinet Office Minister Tom Watson. Internet gurus Tom Steinberg, and Tom Loosemore. Former Number 10 policy advisor William Perrin. All bounced back dazed when they tried shoulder charging the Ordnance Survey’s door, as if tripped up by a canny geographer’s sandal on their run up. So my colleague Tom Chatfield and I decided we that needed to find out exactly how the man who invented the web had managed to reinvent the rules of British data.

The story we uncovered will be on the cover of Prospect magazine’s next issue (out on Thursday 28th January). It is a tale of star power, serendipity, vision, persistence and an almost unprecedented convergence of all levels of government. It is the best sort of policy story: one where the policy works, the good guys win, and public interest is served.

The task of doing the digging fell into two parts. First, we had to talk to Tim Berners-Lee himself—no easy task, given we wrote asking for an interview and received a note back from his organisation saying it was totally impossible. But friends in government put in a good word and doors opened. Just after the New Year, Tom Chatfield (a web geek of sorts) found himself Skype-interviewing Berners-Lee, for more than an hour, about his frenetic six months at the heart of British government.

Talking to the inventor of the web is a dizzying experience, especially when you’re actually using his creation to host the conversation. It’s rather like picking up a book and then leaning over to tell Gutenberg what you think about type, or watching The Wire while sitting next to John Logie Baird. Except, of course, that Tim Berners-Lee is still with us, and, as Tom discovered, takes a restless delight in the future of his creation. Isn’t there a temptation just to sit back, Tom asked, and cruise around the world collecting honorary degrees? The idea didn’t even seem to have occurred. For Berners-Lee, the future of the web is just too important, too urgent and, perhaps above all, too interesting to be neglected.

It was easy to see how this man makes things happen: his viewpoint was never less than global, and his ambition every bit as total as his original vision for “phase one” of the world wide web. Connecting 1.5bn people to each other has been, clearly, only the beginning. There was a touching note of pride in his recollections of family holidays spent in England, finding beaches to visit thanks to closely-clutched Ordnance Survey maps, but this had nothing to do with nostalgia: it was about the power of information, and what he believes can be done if the force of public data is properly unleashed.

For Berners-Lee it was the argument that mattered–as he might say, the need to “get raw data now.” But the context was not easy. Here is how our story begins, from the introduction to the essay:

“It can be tricky to explain why Tim’s work matters so much,” says dotcom entrepreneur turned government adviser Martha Lane Fox. “But the data he has been able to release can reorder the balance of power between the citizen and the state.” Such claims are often made for “e-government,” whose hype is traditionally exceeded only by the price tags attached to the (often disastrous) IT projects undertaken in its name. But Berners-Lee’s work has the potential to be different, relying as it does on an unprecedented combination of technology experts, amateurs and businesses like Dr Foster or Experian to take the new data and present it usefully. This helps people make better decisions, underpins information-age businesses, and—because it to some degree redraws the boundary between people and government—may also change the terms of politics. Yet perhaps the most remarkable fact is that it happened at all. Others have tried to unlock Britain’s data, only to run into walls of official obstinacy, vested interest and political indifference. So how did Berners-Lee do it?

Finding out the answer to this was the real work: the details of the meetings with cabinet ministers, the tense Whitehall battles, the crucial presentation to the Cabinet, and so on. We spoke at length to Berners-Lee’s friend and co-appointee to the data initiative, Nigel Shadbolt, and had carefully worded off-the-record chats and “steers” from officials from many different government departments, political figures and well-connected outsiders. And it was here we found the story.

****

In their exclusive story of how the man who invented the web broke open Britain’s government, only in Prospect on 28th January, James Crabtree and Tom Chatfield reveal:

The inside story on how Tim Berners Lee and Nigel Shadbolt opened up Britain’s secretive, closed data culture

How a single, tragic accident convinced Britain’s top politicians of the wisdom of data-mashing

Why a single Word document sent to “TBL” suddenly made the centre of government stop

How a ministerial trip to see President Obama’s team finally opened the door for Tim Berner-Lee’s vision

How Tim and Nigel both believe their vision for public data could incubate the next “phase 2″ of the web, while also redrawing the boundary between citizens and government

New Prospect poll: The rise of Britain’s liberal “twittering classes”

James Crabtree
birdandt_tcm18-151058

Yes, its yet another poll about twitter

In the new edition of Prospect—in shops tomorrow, but on subscribers’ doormats today—we have a new poll as part of a fun feature about the politics of Twitter. The claim is that Twitter is an oddly liberal tool: a mouthpiece for what we have (not entirely originally, I must admit) dubbed the “twittering classes.”  The killer bit of the poll is where Twitterers lie on our scale of liberal/authoritarian groups—way out to the extreme, see the graphic below. It also has some lighthearted stuff about Twittering figures from history too—more on that later tomorrow. You can find it on page 37 of the print edition, inside Evgeny’s wonderful piece on dictators and the web.

We’ll put all this up on the web soon, along with responses to Evgeny’s thesis from Clay Shirky and others. But given the Guardian and Telegraph did stories on our Twitter poll today I thought I’d just put up the text of press release for people to see. Obviously we’d be grateful if you could tweet this, mentioning @prospect_uk when you do so!

****

PROSPECT PRESS RELEASE: NEW TWITTER POLL

Often seen as little more than a harmless waste of time, the much-hyped online social network site Twitter is increasingly being used as a tool by liberal and left-wing political campaigners. Twitter users are among the most liberal groups in Britain, a new national poll of 2000+ people by Prospect magazine and pollsters YouGov reveals.

Read more »

A sneak preview of December’s Prospect: why dictators love the web

Brian Semple
165_cover

The internet: a dictator's best friend?

When Barack Obama met with Chinese students yesterday in Shanghai, he called for greater internet freedom in China, saying that “the more freely information flows, the stronger the society becomes.”

However, in the upcoming issue of Prospect web guru Evgeny Morozov argues that the west has not only vastly overestimated the power of repressive regimes to censor the internet, but is also misguided in thinking that the internet is good for democracy. Instead, authoritarian regimes are finding new ways to control the web:

The “great firewall of China,” which supposedly keeps the Chinese in the dark, is legendary. Such methods of internet censorship no longer work. They might stop the man on the street, but a half determined activist can find a way round. And more often than not, official attempts to delete a post by an anti-government blogger will backfire, as the blogger’s allies take on the task of distributing it through their own networks. Governments have long lost absolute control over how the information spreads online, and extirpating it from blogs is no longer a viable option. Instead, they fight back. It is no trouble to dispatch commentators to accuse a dissident of being an infidel, a sexual deviant, a criminal, or worst of all a CIA stooge.

The December issue of Prospect is available to buy in shops from Thursday 19th November, and subscribers can read the piece online at www.prospectmagazine.co.uk

Murdoch: readers must pay for web news

Jonna Dagliden
Next summer Murdoch will charge for webnews

Next summer Murdoch will start charging for web news. But will readers cough up?

Media mogul Rupert Murdoch has finally answered the question we all have posed this year: when and where will we have to start paying for online news? “Next summer,” he finally told the Guardian today.

Speculation has been rife on whether charging for online content would be the final solution for newspapers who are struggling to survive. Writing in the June issue of Prospect, Damian Tambini warned that the idea is part of a general shift in the way we are able to access the internet. “It will limit both competition and citizens’ ability to access freely the internet’s treasure trove,” he wrote, adding: “Openness doesn’t just guarantee free speech, it also partly explains recent unprecedented levels of innovation.”

However, it remains to be seen whether Murdoch’s wheeze is workable. Can newspapers take back what readers have become used to consuming for free? Murdoch has said that Newscorp will start charging for news from both the broadsheets and tabloids such as the Sun, celebrity news being of special interest. “When we have a celebrity scoop, the number of hits we get now are astronomical,” he told Guardian. But whether readers will think it’s actually worth shelling out for such scoops when they can now pick up several freesheets full of  gossip daily is another question all together.

The future of search: why don’t you Baidu it?

Tom Chatfield
China's present, the world's future

China's present, the world's future

With the recent news of Google’s travails against censorship in China (its site was blocked for several hours amid accusations that it was helping to “spread pornography” and break China’s strict, if opaque, laws on what is and isn’t legitimate online), it seems a good time to be talking about the future of search. Not the future of global search which, I think we can safely assume, will be dominated by Google for years to come. But the future of an altogether more regional internet. In my science and technology piece this month, I cast an eye over the booming field of regional search: and, in particular, at China, which now boasts both the world’s largest online community – with almost 300m Chinese citizens online – and a regional internet in which Google is but a minor player compared to the might of home-grown Chinese search engine Baidu.

The end of the free internet?

Damian Tambini

Murdoch: the death of free speech?

Murdoch: the death of free speech?

Reports today that News International will move the Sunday Times online to a “paid-for” model are part of a more general shift away from a “free as in free beer” internet, as I mentioned in my article in this month’s Prospect. But do these moves mean we should worry about a move away from a “free as in free speech” internet?

It depends of course whether your definition of free speech includes people’s right to receive and exchange ideas among one another. If it does, the brief period in which newspapers and broadcasters have attempted to gain market share on the internet by offering free access and have funded distribution through advertising has been a boon for free speech. It has been a period of amazingly fast and free exchange of ideas. Facebook, Google and the wider blogosphere have emerged as the major new distribution mechanisms for news and comment and they have enabled us to flag, point, post, tag, “like” and comment freely and quickly. Yet as the newspapers teeter on the edge of collapse, we do have to acknowledge that somebody has to pay for journalism.

Read more »

The dangers of the clickstream

Mary Fitzgerald
Shannon Matthews: a tent pole story

Shannon Matthews: a tent pole story

As advertising budgets are squeezed, the internet may be the only thing that saves the media, reports Andrew Currah in this month’s Prospect. Thanks to web analytics technology, publishers can offer advertisers much better targeted and more effective adverts; and publishers themselves can learn more about what people want to read. This all could potentially make the news becomes more accessible, engaging and relevant.

Read more »