Op-Ed: Seeking Middle Ground In Scrap W/ Scrapers

American and Southwest airlines, two carriers at opposite ends of the spectrum and fierce competitors, have been on the same side of the fence on the issue of screen scraping. They have been the most vocal on screen scraping in general and both maintain that they should have the right to choose who gets access to the data. However, the public needs to be able to make a decision based on the information that is out there.

We have innumerable studies that show that customers are indeed shopping, as is their right. This decision is a singular one for the consumer and not one that should allow another to profit from the information with no regard to the consequences. There is no difference in the virtual world than in the physical one. From our discussions with the airlines, we sense that they would be willing to allow screen scraping to occur—if there was a clear understanding of the value to both sides. Priceline already has come to some accommodation with the private sites that sprung up around it.

Perhaps now is the time for the airlines and the scrapers to come together.

Yet, we have to remember that the economics are not that great. Just as the GDSs learned of the negative cost impact of screen scraping on old green screens, the people who run supplier Web sites now face the same challenge.

Maintaining a Web site does cost money and while the amount is small there is a real cost per "hit" or per query. When a scraper is working, it can add thousands of hits to the Web site, which adds up to real money. Also, the scrapers skew data and their actions result in higher look-to-book ratios than is actually present.

At the heart of the matter is perhaps the fundamental nature of the Internet itself and the fact that there is near perfect information out there.

Consider this:

If AA wins its suit against FareChase permanently, then transparency will take a big knock. This is good for the suppliers but bad for the consumer. If FareChase wins permanently, then every supplier will be much more cautious about the display and distribution of the data. This will result in more private sites and, in our opinion, the consumer will have to make more clicks to get their information.

We would propose, however, that there is a middle ground. The scrapers and spiders should be allowed to continue to have access but that access should be controlled via agreement—that agreement not reasonably withheld.

In other words, if the airlines or any suppliers put the data out through an open Web site, then they cannot unreasonably prohibit the onward distribution of that data. That is providing that the players, such as SideStep, FareChase and even other players like AgentWare, recognize the cost and the ownership of that data.

Good luck to both protagonists!

Timothy O'Neil-Dune, a U.K. native who has worked in travel distribution for nearly 25 years, is the leader of T2Impact, a business development and strategic consulting group.