Archived; click post to view.
Excerpt: In the last post, we talked a bit about the Magestic SEO link analysis site. It gets its data from their own search engine known as Magestic-12. This project started in 2004 and uses distributed computing much like the SETI@home project which analysed data from the Arecibo radio telescope looking for signs of intelligent life in the universe. The main advantage to the distributed computing element of the search engine is that it can crawl the web faster and update its index more frequently. Volunteers can download software that turns their machine into a search engine node. Allowing it to…