As of January 2014, the size of the Internet, that is indexed, consists of more than 2.04 billion pages. As of 2012, Google — the world’s largest index of the Internet — has only indexed a portion of that: approximately 50 billion pages (about 5 million terabytes of data store on more than 900,000 servers). If each web page were a sheet of paper, a print version of Google’s indexed website would be more than 3,156 feet high, slightly taller than El Capitan, the iconic granite monolith in Yosemite National Park in California — now that’s what you call a truly monumental book. So who reads all these websites? Who are the Internet’s most voracious readers?
Most individuals will be surprised to learn that humans do not make up the majority of the readers of the Internet. A study published in December 2013 by Incapsula, an Internet services company, found that only 38.5% of Internet traffic was generated by humans, while 61.5% of Internet users were non-humans; specifically, search engines, bots (or web crawlers), scrapers, hacking tools, spammers, and other impersonators. And perhaps the most disturbing discovery was that 60% of the non-human Internet use was malicious. A year earlier, in 2012, Internet use by humans was 49% and use by bots was 51%. Sadly, bots are the most voracious (and malicious) readers of the Internet, like a horde of vandals, armed with spray paint, scissors, and butane lighters, that are released into the Library of Congress to wreak havoc. What will survive?
Below is the breakdown of human and bot traffic on the Internet:
38.5% Human traffic
61.5% Non-human traffic:
31% Search engines
4.5% Hacking tools
20.5% Other impersonators
For further reading: incapsula.com/blog/bot-traffic-report-2013.html