Hi, I’m Dan Ni, the founder of Scraper API. I created this tool because as a freelance developer I often had web scraping jobs where I would go through the same boiler plate tasks (setting up proxy servers, handling IP rotation, running headless browsers, etc.). I decided to wrap all of this up into a simple to use API so developers could write web scrapers with essentially no setup.
Scraper API is an API that helps developers easily build scalable web scrapers. Unlike proxy providers that just give you a list of IP addresses, Scraper API handles IP rotation, headless browsers, and CAPTCHAs so developers can get the HTML of any web page with a simple API call. The developer experience is our key differentiator over our competitors. All you need to do is give us the URL and we’ll return the HTML of the page.
Our philosophy is to do one thing and do it well; our API has just one endpoint (with a few config parameters you can set), so everything is fairly straightforward and most of our users can get up and running within minutes. Our documentation is available at https://www.scraperapi.com/documentation, we offer 1000 free test requests per month as well as paid plans with higher limits at https://www.scraperapi.com/pricing.
Scraper API is used by over 1000 businesses across many different industries. The primary use cases are ecommerce price monitoring (shopping deals sites or SAAS companies catering to ecommerce clients who want to monitor their competition), SERP monitoring (this is mostly for SEO tools who help their clients monitor their rankings on popular search engines), social media (a number of social media monitoring apps and hedge funds use us to get data from popular social networks), real estate/travel sites (real estate and travel apps both use us to gather listing data across a number of platforms), reviews (companies concerned about their reviews will use us to monitor reviews across a number of different platforms).
We have an affiliate program (https://www.scraperapi.com/affiliates) and we actively reach out to a number of developer blogs who write good tutorials using our tool. We are just now beginning to sponsor a few developer blogs and Youtube channels to get the word out about our tool.
Around 2 billion requests/month.
NodeJS, PostgreSQL, Redis, and Puppeteer for our headless browsers.
We are completely revamping our dashboard to provide better metrics and monitoring.
Stripe for payments, Amazon SES for transactional email, ipinfo.io for IP Address information, LogDNA for logging.
Running an API as a business is great if you’re a developer because you can see exactly how users are using your tool just by checking your logs. This means that your feedback loops are instantaneous and you can get an accurate idea of how your user base as a whole behaves by looking at aggregate metrics. I highly recommend it!
You can always reach us at [email protected].
