August 2014 – Tech Blog

The privatisation of Royal Mail came with a massive Open Data defeat, having the Postcode Address File been left within the assets of the sold company. The Open Data User Group – and many others players in this space – voiced their dissent, but the decision had been taken: PAF had to go with the rest of Royal Mail.

It’s time to move on.

The newly funded Open Addresses project is a great opportunity in this context and a Symposium run by the ODI on August 8th has just reinforced my impression that some of the smartest people in the “Open” community are working incessantly to create a credible alternative to PAF (and to Ordnance Survey’s Address-point). Let’s call this OAF.

The Open Addresses Project represents a great opportunity.

First of all, it’s an opportunity to show that crowdsourcing can be as good as a top-down approach, if not better.

We’ve seen this happening with OpenStreetMap. In many parts of the world, the coverage and accuracy of OSM is not just way beyond that of commercial solutions, but it is kept constantly in check by an army of volunteers and users.

Why hasn’t OSM gone mainstream to show the power of crowdsourcing? Primarily because Average Joe doesn’t get that map means archive of location-based data points. Average Joe reads map and thinks Google Maps. “You can’t use OSM” is a common objection, and this misses the point about OSM.

OAF would incur this danger considerably less. Addresses are readily understandable by people, and there would be no confusion about what the database represents.

In an address file, location is an attribute of the address; whereas in a map, address is a attribute of the location.

There is of course another problem of OSM: it is not perceived as authoritative. I remember a discussion with a travelled contact of mine heading to India and looking for maps in constantly changing area. “Use OpenStreetMap”, I said; “it will never be good enough!”, he replied. Except that when he checked, he found out that OSM had more data than any other map available to him.

Open Addresses could definitely risk a perceived lack of authority. Advocacy will play an important role, together with case studies, independent evaluations, and early adopters, in showing that the Open Address File can become the authority in this space.

Secondly, this is an opportunity for Royal Mail and Ordnance Survey to review their practices about addresses and to improve their own products by providing users with an independent way to assess their quality. It’s maybe an opportunity for them to finally and uncontroversially understand that Openness is a way to increase their business influence – and revenues – and not a way to jeopardise it.

There are unquestionably several challenges ahead, both technical and non-technical.

The technical challenges can be easily enumerated: what is an address? What is the minimum unit of space we want to represent? Where do we stop – street level, floors, flats, units, rooms? None of these will be easy to solve, but the beauty of this project is in its process that intends to bring together addressing experts with users of addresses.

However, considerably more challenging will be the cultural problems children of the monopoly of Royal Mail and Ordnance Survey in this space, the myths about what PAF could deliver and the reliance on restrictively licensed products that might hinder the smooth transition to an open product.

Let me mention some of these cultural challenges:

PAF was built as a way to allow postmen to deliver post; it’s a collection of delivery points, not a way to identify buildings, houses, or business premises, although some users have come up to intend it this way; OAF will need to re-state its goals and make them easy to understand for its users
PAF can be authoritative in the minds of its users, but it’s not as accurate as they generally believe: duplications and errors are rather common; OAF has the opportunity to be more accurate and have more coverage and will need to be up to this for at least a number of clear use cases. Let me quote ODUG’s response to the PAF consultation:

Royal Mail usually states the completeness of PAF (the principal measure of quality) as being in excess of 98%. However, as Royal Mail determines what a delivery point is, no external body can identify missing delivery points to confirm that measure.

evidently, there is no benchmark to assess the quality of PAF; OAF will need to be built in a way that makes this assessment possible and desirable to its users
feedback loops will need to be clear, i.e. how to allow third parties to add addresses into OAF
the NLPG is often hailed as a solution, when in fact is just another face of this problem, coming with restrictive licensing

All of these issues are difficult, but not unsolvable. I believe that early adopters will play an important part in advocating this new data product, showing that it’s not just as good as the existing commercial not-really-open solutions, but better in terms of reliability, accuracy, coverage.

Most use cases of an address file revolve around the function of address lookup. As such, they come with a great feature: they immediately detect if an address is missing or incorrect. Feedback will play a relevant part in the process envisioned to build Open Addresses.

Hence, let me close with an appeal: if you use addresses in your business, for your job, for marketing purposes, keep an eye on this project and start building your services in a way that allows you to use an Open Addresses file.

In 1999, I was 17 and I had been on Internet, as we used to say, for about 3 years then. I was still at school – in Southern Italy – and as part of our curriculum we had to write a monthly essay, having a choice of topics and styles including that of a newspaper article. The only reason why I chose this was that it was the only one for which our teacher would pre-announce the topic (all other being surprise).

Back then, the Yugoslav Wars had “ended”, pretty much, in 1995. No one had ever heard of Kosovo, a small region of what became Serbia, with a relevant Albanian minority. After alleged genocide and the international community condemning Serbia, despite a veto from the UN Security Council, NATO decided to bomb Yugoslavia.

In those years, there was no Twitter, no Facebook, not even Orkut. Actually, not even Google. Blogging was, maybe, something people would write on their home page (mine was provided by my local ISP, and it had a grey background). But the world fascination with the Internet was getting bigger and bigger. People would start talking to random people in public and private chats. Those were the years of ICQ, the beginnings of MP3 as a music medium – when MP3 files could be searched for on Altavista.

My essay about the Belgrad bombing was born on IRC. I joined the channels #kosovo and #belgrad. I started calling random people, declaring my intentions: I wanted to ask about the war, what it was like being there, what were their feelings about NATO, Kosovo, Serbia.

Most chatters, on both sides, were happy someone was asking about them. Some of them were expats trying to get in touch with relatives; some others managed to get an dial-up internet connection in their garage, turned bomb refuge. I spent a number of nights mostly trying to collect balanced views, but more importantly establishing a direct, sometimes emotional connection with people who were living not that far away – my town being about 600km and a fly away from Pristina – and experiencing a terrible situation, in a very polarised way.

IRC helped me gather those view and write my essay. Internet became to me, in the 90s, a superb means of getting to know people who were far away from me, understanding them, perceiving their experience; despite a limited, text-based, medium, that connection could happen. Real-time journalism was a reality; it was by its nature a one-to-one experience, not a collection of tweets on a given hashtag.

We’re moving away from that, and let me be nostalgic: something has got lost. We’re now damn good at aggregating content, but with all of this aggregation we’re probably failing to make that one-to-one connection. The early Internet facilitated that connection; the current, social-based Internet, paradoxically cannot.