The complete automotive resource for buyers, sellers, and owners like you.

Roadfly Corporate Blog

Scraping, the new buzz word. What’s with all this blatant stealing?


Filed under: Opinions & Commentary
Tags:
Written by Charlie Romero      April 6, 2006 , 12:06 pm

First it was mp3 music, then it was Hollywood movies and now it’s editorial content. Editorial content which is blatantly stolen via RSS/XML feeds. What’s with all this stealing? Just because it’s there doesn’t mean it’s free to be stolen.

We recently modified the Roadfly Terms of Service to more clearly explain that commercial businesses can’t take our RSS/XML feeds, automatically scrape out the content they like and then republish or distribute it as their own.

The wild, wild west Internet along with Google’s AdSense program has spawned a cottage industry which builds web sites using other people’s copyrighted content and then republishes it as thir own. These unscrupulous webmasters publish their website and then generate AdSense ad revenue from Google. The Google AdSense example is most typical and most talked about but there are also big businesses that do it. Big businesses that use robot technology to harvest or scrape copyrighted information from our pages. Companies that build entire business models around stealing via RSS or web crawlers. We are working to put a stop to it and so far it looks like our efforts are working.

You’re probably wondering why this topic made it onto the Roadfly Corporate Blog in the first place. Well there are two reasons, one is that it gave me the chance to rant and the other is to help bring the illegal practice more negative attention.

Our software developers (headed by the uber-genius himself Alan Riley) are in the process of enabling RSS/XML feeds for all the content on Roadfly. We first RSS enabled our car classifieds about a year and a half ago. A few months ago we enabled RSS on our car reviews, feature articles and other editorial content. We’ve also RSS enabled our car picture galleries which we include in our car reviews, etc. Over the course of the next few months we’re going to gradually enable RSS/XML feeds of our forums and all of our other content.

Knowing that we are syndicating nearly all the valuable content on Roadfly didn’t sit well with Alan and me. And, you know what our copyright lawyer the famous Steve Trattner had to say. So in addition to RSS enabling all of our content we are building monitoring tools to keep an eye on the content thieves.

The scrapers have been noticed all over the net because they’re doing some pretty bad things. In plain English they’re stealing copyrighted work. They know what they are doing and most of their legal deptartments turn a blind eye. They tell the boss “let’s see how long we can get away with it.”

Well Roadfly for one isn’t going to sit on the sidelines and be a victim. We’re watching the thieves and building walls. Stay tuned for more.