Once a side product of doing business, data has taken the center stage. Web scraping has helped many companies get very data-rich very quickly; but it comes with a risk of disputes and lengthy litigation. Open data offers an oft-overlooked yet powerful way to grow a business - without the risks and restrictions of web scraping.
Classic case of AirBnB
Today a company worth billions, AirBnB had humble beginnings. At one point, its founders were forced to sell cereal boxes to finance the startup. It was not until their ingenious decision to make use of Craigslist that AirBnB really took off.
Being a two-sided platform, AirBnB faced the old chicken-and-egg problem: to get guests, it had to have hosts, and vice versa. No one would want to use an empty platform. So, the founders came up with a brilliant idea to tap into Craigslist's massive listing database and kickstart their own service.
First, they reversed engineered Craigslist's ad-posting form and created a bot to automatically fill it with data. Then, they offered each new user to re-post their AirBnB listings on Craigslist, promising more earnings. Such ads were often better-made, so they attracted new users to the platform.
The second measure was to regularly scrape Craigslist for new postings using automation tools. As soon it found a new post, AirBnB would send an automated email to the author, inviting him or her to the platform.
Of course, doing so was hardly ethical. It also infringed on Craigslist's terms of use. But it gave AirBnb the kick it needed to catapult the startup into a billion-dollar unicorn.
Legality of web scraping
AirBnB is a classic example, but it's far from the only one. There are companies whose whole business models rely on data they collect from other businesses. They use specialized tools - web scrapers - to crawl and pull relevant information. This arrangement is very one-sided, and it often results in enmity.
You don't have to look far to find such companies. Most of the travel fare aggregation websites stand on data they scrape from airlines. Take Expedia, a multi-billion business. It started by collecting ticket prices and offering the best deals to holiday-goers. Over the last few years, Expedia and several other aggregators have battled Ryanair over the right to scrape the airline giant's data.
HiQ is another example of a data-based service gone to court. Its business model revolves around scraping LinkedIn profiles. The company then creates analytics tools to predict if an employee might quit or if he or she needs extra training. LinkedIn caught wind of such use of its data, interpreted it as a threat to its business, and took action to stop it.
In the end, Expedia settled out of court and HiQ won the lawsuit. But these litigations have revealed the potential risks of piggybacking on others. Data is a valuable asset, and companies are reluctant to give it away for free, even if it is publicly available.
New avenues in open data
The lesson is clear: just because some data is public, you are not free to use it however you like. If you do, it can get you into trouble. But another type of data exists, and it was designated for this very purpose - free and unrestricted use. It is called open data.
Open data is depersonalised information made available for anyone to access and use. It includes geospatial and mapping data, national statistics data, transport data, weather and environment data, and other datasets. You can usually find it on governmental websites, though some private companies like OpenCorporates publish open information as well.
Institutions promote open data as a way to foster innovation, improve efficiency, and grow businesses. But for some reason, the private sector is reluctant to embrace it. Perhaps the entrenched belief equating data value with cost is at fault. In any case, open data presents little-explored growth hacking opportunities.
Open data in practice
With some ingenuity, companies can benefit from open data in multiple ways. They can start a new product or service. They can improve existing offerings. Or they can get better intelligence on competitors and their own customers.
Open data is very well-suited for companies with retail presence. By combining geo-based datasets (such as retail points, census, urbanity, and other information) they can extract valuable insights on new store locations, delivery optimization, and customer habits. For example, British retailer Tesco predicts purchasing patterns based on openly-available weather data.
Open data can also assist e-businesses. Job-search platform Adzuna uses open labor market data to make elaborate salary statistics. They include salary changes, vacancy numbers, and market trends. This allows Adzuna to differentiate itself in a competitive market and attract users. It also brings extra revenue in data sales.
Open data has found particular success in mobility-as-a-service platforms. Such platforms combine many data sources-Google's mapping service, ride-sharing integration, public transport schedules-to give users an all-around mobility ecosystem. A large part of their appeal are live transit feeds. They allow algorithms to make more accurate route suggestions and commuters to better plan their journeys.
Many opportunities ahead
Data has become a prime resource, and its importance is only bound to grow. Open data offers untapped growth opportunities for any business. Best of all, it comes without the risks of unauthorized web scraping.
