This process is called Web crawling or spidering. Many sites, in particular search engines, use spidering as a means of providing up-to-date data. Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for sending spam).
correct definition?
Anyways that friend of yours must have been hardcore and google must have been upset about that thing your friend did. I told a couple of my friends at school about that and they lol'd.
When you get older you may end up working for a company. There will probably be lots of people there. You will be pretty lucky if even 1% of them are your "friends".
In this case, it was another contractor I only barely knew... and he was working on some other project unrelated to mine.
Also, I don't think we were causing serious harm to google anyway... just hammering their local node so they asked us to stop.
