Web Crawling is a perfect application for Neocortix Scalable Compute instances. We have proven Apache Nutch as a Web Crawling client running on Scalable Compute Instances, and we can currently run at a scale of 1,000 devices.

Our mobile devices are located in people's homes, with residential IP addresses, so there is no need for additional Proxies.

Because of our low cost per instance (about 1/2 the cost of Data Center instances), Neocortix can do Web Crawling in the same time as other Cloud providers at about 1/2 the cost.