Category: open source

Categories
geo open data open source

Open Addresses: a great opportunity

The privatisation of Royal Mail came with a massive Open Data defeat, having the Postcode Address File been left within the assets of the sold company. The Open Data User Group – and many others players in this space – voiced their dissent, but the decision had been taken: PAF had to go with the rest of Royal Mail.

It’s time to move on.

The newly funded Open Addresses project is a great opportunity in this context and a Symposium run by the ODI on August 8th has just reinforced my impression that some of the smartest people in the “Open” community are working incessantly to create a credible alternative to PAF (and to Ordnance Survey’s Address-point). Let’s call this OAF.

The Open Addresses Project represents a great opportunity.

First of all, it’s an opportunity to show that crowdsourcing can be as good as a top-down approach, if not better.

We’ve seen this happening with OpenStreetMap. In many parts of the world, the coverage and accuracy of OSM is not just way beyond that of commercial solutions, but it is kept constantly in check by an army of volunteers and users.

Why hasn’t OSM gone mainstream to show the power of crowdsourcing? Primarily because Average Joe doesn’t get that map means archive of location-based data points. Average Joe reads map and thinks Google Maps. “You can’t use OSM” is a common objection, and this misses the point about OSM.

OAF would incur this danger considerably less. Addresses are readily understandable by people, and there would be no confusion about what the database represents.

In an address file, location is an attribute of the address; whereas in a map, address is a attribute of the location.

There is of course another problem of OSM: it is not perceived as authoritative. I remember a discussion with a travelled contact of mine heading to India and looking for maps in constantly changing area. “Use OpenStreetMap”, I said; “it will never be good enough!”, he replied. Except that when he checked, he found out that OSM had more data than any other map available to him.

Open Addresses could definitely risk a perceived lack of authority. Advocacy will play an important role, together with case studies, independent evaluations, and early adopters, in showing that the Open Address File can become the authority in this space.

Secondly, this is an opportunity for Royal Mail and Ordnance Survey to review their practices about addresses and to improve their own products by providing users with an independent way to assess their quality. It’s maybe an opportunity for them to finally and uncontroversially understand that Openness is a way to increase their business influence – and revenues – and not a way to jeopardise it.

There are unquestionably several challenges ahead, both technical and non-technical.

The technical challenges can be easily enumerated: what is an address? What is the minimum unit of space we want to represent? Where do we stop – street level, floors, flats, units, rooms? None of these will be easy to solve, but the beauty of this project is in its process that intends to bring together addressing experts with users of addresses.

However, considerably more challenging will be the cultural problems children of the monopoly of Royal Mail and Ordnance Survey in this space, the myths about what PAF could deliver and the reliance on restrictively licensed products that might hinder the smooth transition to an open product.

Let me mention some of these cultural challenges:

  • PAF was built as a way to allow postmen to deliver post; it’s a collection of delivery points, not a way to identify buildings, houses, or business premises, although some users have come up to intend it this way; OAF will need to re-state its goals and make them easy to understand for its users
  • PAF can be authoritative in the minds of its users, but it’s not as accurate as they generally believe: duplications and errors are rather common; OAF has the opportunity to be more accurate and have more coverage and will need to be up to this for at least a number of clear use cases. Let me quote ODUG’s response to the PAF consultation:

Royal Mail usually states the completeness of PAF (the principal measure of quality) as being in excess of 98%. However, as Royal Mail determines what a delivery point is, no external body can identify missing delivery points to confirm that measure.

  • evidently, there is no benchmark to assess the quality of PAF; OAF will need to be built in a way that makes this assessment possible and desirable to its users
  • feedback loops will need to be clear, i.e. how to allow third parties to add addresses into OAF
  • the NLPG is often hailed as a solution, when in fact is just another face of this problem, coming with restrictive licensing

All of these issues are difficult, but not unsolvable. I believe that early adopters will play an important part in advocating this new data product, showing that it’s not just as good as the existing commercial not-really-open solutions, but better in terms of reliability, accuracy, coverage.

Most use cases of an address file revolve around the function of address lookup. As such, they come with a great feature: they immediately detect if an address is missing or incorrect. Feedback will play a relevant part in the process envisioned to build Open Addresses.

Hence, let me close with an appeal: if you use addresses in your business, for your job, for marketing purposes, keep an eye on this project and start building your services in a way that allows you to use an Open Addresses file.

Categories
gov open data open source policy Web 2.0

Making Open Data Real, episode 2: the consultation

This is my response to the Open Data Consultation run by Cabinet office:

“My name is Giuseppe Sollazzo and I work as a Senior Systems Analyst at St. George’s, University of London, dealing with projects both as a consumer and a producer of Open Data. In one previous job, I was dealing with clinical data bases so I would say I developed a certain feeling for issues around the topic of this consultation both from a technical and policy-based perspective.

 

An enhanced right to data

I believe this is the crucial point of the consultation: the Government and the Open Data community need to work side by side in developing a culture that fosters openness in data. The consultation asks specifically what can be done to ensure that Open Data standards are embedded in new ICT contracts and I think three important points need to be made:

1) independent consultants/advisors need to be taken on board of new ICT projects when the tendering process is started; such consultants need to be recognised leaders of the Open Data community and their presence should ensure the project has enough drive in its Open Data aspects.

2) Open Source solutions need to be favoured over proprietary software. There are Open Source alternatives to virtually any software package. Should not this be available, a project should be initiated to develop such a solution in-house with an Open Source licence. Albeit not always free, Open Source solutions will offer a standard solution for a lower price, and will create possibilities for resource-sharing and business creation.

3) ICT procurement needs to be made easier. Current focus of ICT procurement in the public sector is mostly on the financial stability of the contractor. I argue it should rather be on reliability and effectiveness of the solution proposed. Concentrating the focus on financial stability is a serious mistake, mainly caused by the fact that contractors will develop proprietary solutions; a bankruptcy becomes a terrible risk because of the closedness of the solution; because no other company would be able to take it where the former contractor left; hence the need of strict financial requirements in the tenders. I object to this. In my view, relaxing the financial requirements and moving the focus to the quality of the solution, its openness, its capability to create an ecosystem and be shared, its compatibility with open standards, will improve the overall effectiveness of any ICT solution. Moreover, should the main contractor go bankrupt, someone else will be able to take their place, provided the solution was developed according in the way I envision: consequently, no need for strict financial requirements.

 

Setting Open Data Standards

As I have already stressed in the previous paragraph, the Government will need to change its rules of access to ICT procurement. Refocusing the attention to openness, standards, ability to re-share the software, is the way to go to start setting a new model in the Open Data area. Web standards can be used and they can represent an example to follow to create new data standards. Community recognised leader can help in this process.

 

Corporate and personal responsibility

It is absolutely important that common sense rules are established and make into law. The goal of this is not to slow operations down, but to ensure that the right to data mentioned earlier on is actually enforced.

The consultation asks explicitly how to ensure the commitment to Open Data by public sector bodies. I believe that, despite many people feeling that the Government should “stay away”, there is a strong need for smart, effective regulation in this area. Think about the Data Protection and the Freedom of Information Act. Current legislation requires many public bodies to deal with data-sensitive operations, and most do so by having a Data Protection Officer and a Freedom of Information Officer. I believe that an Open Data Officer should operate in conjunctions with these two, and that this would not require many more resources than already allocated. The Open Data Officer should drive the publication of data, and inspire the institution they work for to embrace the Open Data culture.

The Government should devolve its regulatory powers in this area to an independent authority to be established to deal with such regulatory issues. I envision the creation of Ofdata on the model of Ofcom for communication and Ofsted for education.

 

Meaningful Open Data

A lot of discussions have been going on about the issue of data quality. Surely, the whole community aims for data to be informative, high-quality, meaningful and complete. Unfortunately, especially at the beginning of the process, this is hard to reach.

I think that lack of quality should never be a reason for publication to be withheld: where data is available, it should be published. However, I also believe that quality is important and that is why the Government should publish datasets in conjunction with a statistical analysis and independent review (maybe run by the authority I introduced in the previous paragraph) that assesses the quality of the dataset. This should serve two goals: firstly, it would allow open data consumers to deal with error and interpretation of data; secondly, it would help the open data producer to investigate problems in the process leading to the publication and setting goals in its open data strategy.

The final outcome of this publish-and-assess procedure would be a refined publication process that informs the consumers and the public about what to expect. Setting a frequency of update should be part of this process. Polishing the data should not: data should always be made available as it is, and if deemed low quality it should be improved at the next iteration.

There are questions about how to prioritise the publication of data. I believe that in this respect, and without missing the requirements of the FoIA, the only prioritisation strategy should be requests numbers: the more a dataset the public requests, the higher priority it should be given in being published, improved, updated.

 

Government sets the example

I think the Government is doing already a good job with this Open Data consultations, and I hope it will be able to take the lessons learnt and develop legislation accordingly.

Unfortunately, in many areas of the public sector there is still a “no-culture” responsible for data not to be released, Freedom of Information requests going unanswered, and general hostility towards transparency. I have heard a FoI officer commenting “this is stuff for nerds, we don’t need to satisfy this kind of need” to Open Data requests. This is a terrible cultural problem preventing a lot of good to be done.

I believe that the Government should set the example by reviewing and refining its internal procedures for the release of data and responding to FoI requests in a more simple, compassionate way, stressing collaboration with the requestor rather than antagonism.

Moreover, it should be the Government’s mission to organise workshops and meetings with Open Data stakeholders in the public sector, to try and create a deeper perception of the issues around Open Data and its benefits. Being on http://data.gov.uk should be standard for any public sector institution, and represent an assessment of their engagement with the public.

 

Innovation with Open Data

The Government can stimulate innovation in the use of Open Data in some very simple way. Surely it can speed up awards and access to funding to individuals and enterprises willing to build applications, services, and businesses around Open Data. This should apply to both for-profit and not-for-profit ventures, and have as only discriminating factor the received social benefit to their communities or to the wider public.

The most important action the Government can take to stimulate innovation is, however, simplification of bureaucracy. Making Company Law requirements easier to satisfy, as we have already discussed for ICT procurement, is vital to bring ideas to life quickly. Limiting legal liability for non-profit ventures is also a big step ahead. Funding and organising “hackathons”, barcamps, unconferences, and any other kind of sponsored moment where developers, policy makers, charities, volunteers can work together, is also a very interesting way of pushing innovation and making it happen.

 

Open data offers an amazing opportunity of creating “improvement-by-knowledge”. Informed choice, real time analysis, accurate facts, can all be part of a new way of intending democracy and innovation, and the UK can lead the way if its leaders will be able to understand the community and provide it with the appropriate rules that make its tools work and the results happen. This way, we will have a situation where services can be discussed and improved, and public bodies can have a chance to adjust their strategy; where citizens can develop their ideas, change the way they vote, take their leaders to account; and, as a result, communities can work together, and society can be improved.“

Categories
geo gov mobile my projects open data open source policy

Outreach and Mobile: opening institutions to their wider community

[Disclaimer: this post represents my own view and not that of my employer. As if you didn’t know that already.]

Do the words “mobile portal” appeal to you?

I have been working extensively, with a small team, to launch St. George’s University of London‘s mobile portal since last January after we decided to go down the road of a web portal rather than that of a mobile app. The reason for this choice is pretty clear: despite the big, and growing, success of mobile apps, we didn’t want to be locked in to a given platform or to waste resources on developing for more platform. Being a small institution it’s very difficult to get resources to develop on one platform, even less on multiple ones. We also wanted to reach more and more users, and a mobile portal based on open, accessible, resources made perfect sense.

As many of the London-based academic institutions, St. George’s needs to account for two different driving forces: the first is that as an internationally renowned institution it needs to approach students and researchers all over the world; the second is that being based in a popular borough it is part of the local community for which it needs to become a reference point, especially in times of crisis. Being a medical school, based in a hospital and a quality NHS health care structure, emphasizes a lot the local appeal of this institution.

This idea of St. George’s as an important local institution was one of the main drives behind our mobile portal development. We surely wanted to provide a good, alternative, service to our staff and students, by letting them access IT services when on the move. However, the idea of reaching out to people living and working around us, to get St George’s better known and integrated within its own local community, lead us to a thriving experience developing and deploying this portal. “Can we provide the people living in Tooting, Wandsworth, and even London, with communication tools to meet their needs, while developing them for people within our institution?” we asked ourselves. “Can we help people find more about their local community, give them ideas for places to go, or show them how to access local services?“.

This coalition government had among its flagship policy that of a “Big Society”, having the aim “to create a climate that empowers local people and communities”. Surely a controversial topic, nonetheless helpful to rediscover a local role for institutions like us to get them back in touch with their own local community, which in some case they had completely forgotten.

In any London borough there are hospitals, universities, schools, societies, authorities. No matter their political affiliation, if each of these could do something, they would improve massively the lives of the people living within their boundaries. Can IT be part of this idea? I think so. I believe that communication in this century can and does improve quality of life. If I can now just load my mobile portal and check for train and tube times, that will help me get home earlier and spend more time with my family. If I can look up the local shops, it will make my choices more informed. It might get me to know more local opportunities, and ultimately to get me in touch with people.

Developing this kind of service doesn’t come with no effort. It required work and technical resources. We thought that if we could do this within the boundaries of something useful to our internal users, that effort would be justified, especially if we tried to contain the costs. With this view in mind, we looked for free, open-source, solutions that we might deploy. Among many frameworks, we came across Mollyproject, a framework for the rapid development of information and service portals targeted at mobile internet devices, originally developed at Oxford University for their own mobile portal. When we tried it for the first time, it was still very unstable and could not run properly on our servers. But we found a developers community with very similar goals to ours, willing to serve their town and their institution. We decided to contribute to the development of the project. We provided documentation on how to run the Molly framework on different systems, and became contributors of code. Molly was released with its version 1 and shortly afterwards we went live.

Inter-academic collaboration has been a driving force of this project: originally developed for one single institution, with its peculiar structure and territorial diffusion, it was improved and adapted to serve different communities. The great developments in the London Open Data Store allowed us to add live transport data to the portal, letting us have enthusiastic reactions from our students, and these were soon integrated in the Molly project framework with great help from the project community. I think this is a good example of how institutions should collaborate to get services running. A joint effort can lead to a quality product, as I believe the Molly project is.

The local community is starting to use and appreciate the portal, with some great feedback received an the Wandsworth Guardian reporting about a “site launched to serve the community”. I’m personally very happy to be leading this project as it is confirming my idea that the collaborative and transparent cultures of open source and open data can lead to improved services and better relationships with people around us, all things that will benefit the institutions we work for. The work is not complete and we are trying to extend the range of services we offer to both St. George’s and external users; but what we really care and are happy about is that we’re setting an example to other institution of how localism and a mission to provide better services can meet to help build better communities.