recommender systems, Web 2.0

Does the world want recommendations?

NewScientist reports on April 30th that Futureful, a Finnish start-up, is building a predictive iPad based search engine that will use a recommender system. By harvesting information from social feeds from Facebook, Twitter, etc…, its algorithm take the topics that are trending, it analyses the users’ interests and behaviour, and recommends new topics that might interest them.

Eric Schmidt is also quoted as having said “The ability to tell me things I didn’t know but am probably very interested in is the next great stage of search“.

I am possibly cynical about this topic and have extensively blogged (Who wants to be recommended?, May 2009) about the problem of appropriate recommendations and the ability to surprise of such systems.

The problems I see relate to how you are supposed to evaluate a system whose task is to generate surprising recommendations. Especially in academic research, the success of a recommendation engine is traditionally evaluated using a very simple metric: take a list of users choices on the given domain, hide a number of entries, check if the recommender system returns them upon analysing the remaining ones. Straightforward, although several other metrics have been proposed.

Now, how are you supposed to evaluate a system that doesn’t have a reference list? We can surely think of many metrics, some of them quantitative, some of them qualitative (or even social-based):

  • the probability a user follows the suggested link
  • the strength of the trust feeling towards the recommender
  • the fact that a user suggests the recommender system to other users …

However, a metric needs to be meaningful and qualitative metrics often lack this meaningfulness. If I’m a user and I want to be surprised, I will be probably following any random link. I often do that in what I call my serendipitous Wikipedia crawls. My favourite recommender system is, above all, Twitter: I only follow people that make me learn something interesting. Not one of the people that Twitter’s “Who to follow” system recommended me was relevant to me.

So I am a bit confused: what exactly a predictive search engine is really trying to achieve?

geo, geomob, mobile, Web 2.0

GeoMob, 12 May 2011

A good level of participation for yesterday night GeoMob. Despite two speakers’ defections we had a well balanced schedule (one big company, one researcher, one startup) and a rich Q&A session. Here’s my usual summary with some thoughts embedded.

Microsoft Bing Maps, by Vikas Arora (@vikasar), Solution Sales Specialist
General show-case talk as we often have from big companies. However, some interesting products seem to be coming out of the Microsoft pipeline, especially StreetSlide and the partially related Photosynth. Some awesome novelty (although not immediately usable) like the amazing live Augmented Reality video stream on a static image view. I’m not totally sure the GeoMob crowd is the right one to show AR ;-)

There was some good debating about updating StreetSlide imagery, thanks to a question by Ollie. This is a well known problem in Google StreetView, especially in busy London High Streets where shops sometimes change hands multiple times in a year. As a result, by the time StreetView imagery has reached Google’s servers it displays a vintage version of reality. Vikas claims that by partnering with Navteq they will be able to update images every 4-6 months.

Vikas earns the best quote of the night award: “I can’t say much about Nokia except that it’s good for us”.

Mapping Surnames Geographically, by James Cheshire (@spatialanalysis), UCL Geography
I was absolutely fascinated by James’ work upon discovering it on the National Geographic Magazine some months ago. The general subject of this talk is showing how surname origins and popularity can be displayed on a map. Two works were presented about surnames in the US and in London.

The talk and the Q&A session highlighted both the power of a map to show surnames but also its limitations. There are obvious problems of visualization: short and long surnames being displayed in different size, choice of colours, positioning, density, granularity.

Although the map itself is a beautiful item, I think that its dynamic version, able to show the nth most popular surname,┬áis more useful, but only if used… dynamically. What I mean is that in places that are true melting pots like London what it’s interesting is not what surname or surnames are the most popular, but rather what’s the distribution of names of a certain origin in a given place. In other words, given the assumption that certain surnames can be related to certain communities, it’s interesting to see that the first five most popular in a given area are sometimes from five different origins.

James was open about the issues of visualising surnames this way, especially about how to treat granularity (e.g. the Irish community in New York is not as big as it would be). There is lot of work to do in this area and a map is only the tip of the iceberg of research, development, coding, and imagination.

Introducing Eeve, by Jan Senderek (@jansenderek)
Impressive UI analysis for this young start-up whose goal is to let people have fun creating and sharing events. Jan, their CEO, delivered a very interesting talk about how UI can lead to a great mobile application. Their strategy of “mobile first, then web” is interestingly different by that of many other startups around. Event creation and sharing seems to have a mind-boggling peculiarity: initially, events will need to be created in the place where they will be held and shared immediately. No forward planning allowed, which sounds strange but might capture the fantasy of party goers. They plan to extend the service to let event organisers create entries.

The (long) Q&A session seemed critical but was truly interested. First of all, turning myself into the bad guy, I asked what makes them different from their competitors. I’ve attended GeoMob since 2009 and this is at least the third company introducing a similar service, and their unique selling point is not extremely clear. Surely, UI seems to be really good for their app, but is that enough to get to that critical mass of users needed to succeed?

Secondly, the business model seemed not very well defined. Although as any stealth startup Eeve wouldn’t probably disclose too much about it, the general perception was that they need to think about it a bit more accurately, and Jan admitted that.

However, I also have the general impression that small companies presenting at GeoMob (not just Eeve) tend to come just with their shiny iPhone application rather than with the backstage work which might be of great interest. This also gives the wrong impression that most of them are trying to monetise upon nothing more than a mobile app. As it turns out, one of the other LBS introducing at GeoMob a similar event-based app was also selling a CRM system to event organisers which is where their main revenue stream comes from. None of this was mentioned at the presentation and we were left wondering with the same questions.

I won’t mention all the discussions about stalking and privacy: we’ve done that for all companies providing LBS, so nothing new from that perspective. But it’s always good to have our @StevenFeldman pointing that problem out.

To be honest, I’m curious about Eeve and will probably try it out (paying attention to privacy, of course :P). It would be nice to have a report on how many users join the system and especially their B2B strategy.
Maybe for a next GeoMob?