PYTHON SDK

Python SDK for This Web Scraper API

Async-first, fully type-hinted client with automatic pagination, smart rate-limit handling, and built-in retries. Handwritten on top of the generated core, not just an auto-generated wrapper.

Python 3.10+Handwritten typed clientOpenAPI-generated core
pip install scrapebadger uv add scrapebadger
QUICK START
import asyncio
from scrapebadger import ScrapeBadger

async def main():
    async with ScrapeBadger(api_key="your-api-key") as client:
        user = await client.twitter.users.get_by_username("elonmusk")
        print(f"{user.name} has {user.followers_count:,} followers")

        async for tweet in client.twitter.tweets.search_all(
            "python programming", max_items=100
        ):
            print(f"@{tweet.username}: {tweet.text[:80]}...")

asyncio.run(main())

What's Included

⚡

Async-First

Built on httpx & asyncio for high-performance concurrent scraping.

✓

Full Type Hints

Pydantic models for every response, IDE autocomplete on the full API surface.

↻

Auto Pagination

Async iterators like search_all() handle large datasets without manual cursor management.

⟳

Smart Retries

Automatic exponential backoff on transient errors, with console warnings on each retry.

◎

Every Product

Twitter, Google, Amazon, eBay, Vinted, Reddit, TikTok, YouTube, web scraping, and more.

⇄

WebSocket Streaming

Real-time streaming with auto-reconnect, plus webhook signature verification.

Python SDK Questions

How do I authenticate?+

Pass your API key directly, ScrapeBadger(api_key="sb_live_..."), or set the SCRAPEBADGER_API_KEY environment variable and omit the argument.

Does it support synchronous code?+

The SDK is async-first, built on asyncio and httpx. For synchronous use, wrap calls with asyncio.run() as shown in the quick start.

How does pagination work?+

Methods like search_all() return async iterators that page automatically, pass max_items to cap how many results you pull without manually tracking cursors.

Where's the source code?+

Open source under the ScrapeBadger GitHub organization, generated from the same OpenAPI spec as every other language SDK, with Python and Node.js additionally shipping as handwritten, backward-compatible clients.

Start Building in Minutes

1,000 free credits, no credit card required. Same key works across every SDK, MCP, and the CLI.