Scrapy Pricing, Reviews, Pros and Cons in 2026
Scrapy is free to download. It's not free to run. That distinction trips up every team that picks Scrapy for a production project and then discovers the proxy bills, the VPS costs, and the developer hours spent babysitting brittle spiders.
Here's a full breakdown so you know what you're actually signing up for.
What Is Scrapy?
Scrapy is an open-source Python framework for crawling websites and extracting structured data, maintained by Zyte along with 500+ other contributors. It has 55.1k GitHub stars and powers an estimated 34% of production scraping projects. The framework is built on Twisted for async I/O, giving it high throughput on static pages - but "open-source" doesn't mean "free to operate."
The Quick Version
- Scrapy framework: Free forever. No license, no usage limits.
- Production TCO: $150-$500/mo once you add proxies, hosting, and dev time.
- Scrapy Cloud: Free tier with 1 low-resource unit; paid units at $9/mo each.
- Zyte API: From $0.06 per 1,000 successful HTTP responses with a commitment plan.
Full Pricing Breakdown
The Framework Itself
Zero cost. MIT-licensed with no commercial restrictions. Run it on a $5 VPS or a fleet of servers - the framework will never send you an invoice.
Scrapy Cloud
Zyte offers Scrapy Cloud for teams that don't want to manage infrastructure:
| Tier | Price | Retention | Runtime |
|---|---|---|---|
| Free | $0 | 7 days | 1 hour max |
| Paid | $9/mo per unit | 120 days | Unlimited |
The free tier works for testing and includes one low-resource unit with half the resources of a regular unit. Most teams running multiple spiders land on 2-5 paid units. Worth noting: Zyte's commercial services fund Scrapy's ongoing open-source maintenance, which matters when you're evaluating long-term viability.
Zyte API
Zyte's managed extraction service handles anti-bot challenges automatically. Pricing scales by site difficulty and commitment level:
| Commitment | HTTP (Tier 1-5) | Browser (Tier 1-5) |
|---|---|---|
| Pay-as-you-go | $0.13-$1.27/1K | $1.01-$16.08/1K |
| $100/mo | $0.10-$0.95/1K | $0.75-$12.00/1K |
| $200/mo | $0.08-$0.76/1K | $0.60-$9.60/1K |
| $500/mo | $0.06-$0.61/1K | $0.48-$7.68/1K |
New accounts get $5 in free credit. On Standard plans, pay-as-you-go caps at $100/mo in spending; commitment plans raise that to $200-$1,000/mo. Standard plans are rate-limited to 2,000 RPM. Zyte only charges for successful responses - failed and rate-limited requests don't cost you anything.
The Hidden Costs Nobody Budgets For
Here's the thing: the framework is free, but everything around it isn't.
Proxies eat the biggest chunk. Residential proxies run $3-$10/GB; datacenter proxies are cheaper at roughly $0.60-$0.90/GB but get blocked faster. In our experience, proxy costs are the line item that surprises teams most - a single JS-heavy page can weigh 2-5MB, and those gigabytes add up fast when you're crawling thousands of pages daily.
Infrastructure means a VPS at minimum, typically $20-$200/mo depending on crawl volume.
Developer maintenance is the cost nobody budgets for. Scrapers break when sites change layouts or rotate anti-bot measures. One practitioner on r/webscraping reported running 10k-30k articles/day at roughly $150/mo - but estimated the equivalent via paid APIs would cost $3k-$4.5k/mo. The labor to keep those spiders running? That's a separate line item entirely.
You're spending $150-$500/mo on proxies, hosting, and maintenance just to scrape contact data that still needs verification. Prospeo gives you 300M+ profiles with 98% email accuracy and 125M+ verified mobiles - no spiders to maintain, no anti-bot fights, no proxy bills.
Trade your scraping infrastructure for $0.01 per verified email.
How Scrapy Compares
| Solution | Starting Price | Best For |
|---|---|---|
| Scrapy (self-hosted) | Free + $150-$500/mo | Large-scale static scraping |
| Scrapy Cloud | Free / $9/mo per unit | Hosted Scrapy without DevOps |
| Zyte API | $0.13-$16.08/1K reqs | Managed anti-bot extraction |
| Apify | Free / $29-$99/mo | Pre-built scraper marketplace |
| ScraperAPI | Free (1K calls) / $49-$149/mo | Proxy rotation as a service |
| Octoparse | $119/mo | No-code visual scraping |
Reviews: Pros and Cons
Pros:
- Async throughput is unmatched for static content. Scrapy's Twisted architecture outperforms browser-based tools by orders of magnitude on server-rendered pages. Roughly 34% of production scraping projects run on frameworks like Scrapy, and that adoption isn't accidental.
- Massive ecosystem. 55.1k GitHub stars, 500+ contributors, and a decade of Stack Overflow answers mean you'll rarely hit a truly unsolved problem.
- Active development. Version 2.13 introduced
async def start()as a modern replacement forstart_requests(), and the 2.14 cycle keeps pushing forward. - Total control. You own the infrastructure, the pipeline, and the logic. No vendor lock-in, period.
Cons:
- Steep learning curve. Reddit sentiment is consistent here - Scrapy feels "restrictive" and unintuitive for developers used to simpler request/response patterns. The framework's architecture is powerful but opinionated, and that opinion doesn't always match yours.
- JavaScript is a pain point. Scrapy doesn't render JS natively. You need scrapy-playwright or Splash, both of which kill the speed advantage that made you choose Scrapy in the first place.
- Maintenance scales with your targets. Every anti-bot update and layout change means spider maintenance. At scale, this becomes a part-time job.
- Default settings mislead newcomers. New project templates ship with
DOWNLOAD_DELAY = 1andCONCURRENT_REQUESTS_PER_DOMAIN = 1- far more conservative than the framework's built-in defaults of 16 concurrent requests and 8 per domain. Teams that don't tune these wonder why crawls take forever.
Let's be honest: if your deal sizes are sub-$10k, you almost certainly don't need Scrapy-level infrastructure. A managed API or pre-built scraper will cost less than the developer hours you'll burn on maintenance.
Who Should Use Scrapy
Use Scrapy if you're building a large-scale pipeline crawling static pages, you have Python developers on staff, and you're comfortable owning infrastructure. Before writing a single spider, check whether the site has a hidden API - many SPAs expose JSON endpoints that a simple requests call can hit without any scraping framework at all. If you're using scraping to support outbound, it’s also worth reviewing web scraping lead generation tradeoffs and workflows.
Skip Scrapy if your targets are JavaScript-heavy SPAs. You'll spend more time fighting Playwright integration than actually extracting data.
If you're building Scrapy pipelines just to find B2B emails and phone numbers, you're overengineering the problem. Prospeo's database covers 300M+ professionals with data refreshed every 7 days - not scraped once and left to rot. 98% email accuracy, 30% mobile pickup rate, zero maintenance.
Stop scraping for data that already exists in a verified database.
FAQ
Is Scrapy completely free?
The framework is free and MIT-licensed. Production scraping requires proxies ($3-$10/GB), hosting ($20-$200/mo), and developer maintenance - bringing real costs to $150-$500/month for most teams.
Can Scrapy handle JavaScript-heavy websites?
Not natively. You'll need scrapy-playwright or Splash for JS rendering, which adds complexity and significantly reduces Scrapy's speed advantage. For JS-heavy targets, Playwright standalone or a managed API is often simpler.
What's a better option if I just need business contact data?
Scraping for contact data is slow, legally risky, and returns unverified results. A dedicated B2B data platform like Prospeo delivers 143M+ verified emails at 98% accuracy with 125M+ direct dials - no infrastructure, proxies, or maintenance required. The free tier includes 75 emails/month.