How to Solve CAPTCHAs in httpx and aiohttp (No Browser)

An httpx captcha flow is three steps and about twenty lines, and the reason people get it wrong has nothing to do with the solver. There is no browser in this stack. Nothing runs the site’s JavaScript, so nothing creates the hidden field the token is supposed to land in, and nothing submits the form for you. You do all three yourself: read the sitekey out of the HTML, solve it, then send the token back as an ordinary form field on the same client that fetched the page.

What you need

• Python 3.10 or newer, and either httpx or aiohttp. The CapSkip Python SDK works with both.
• The URL of the protected page. You do not need the sitekey in advance, because step one reads it.
• CapSkip running in Local mode when the script and the solver share a machine, or in Server mode when they do not. Both are described under connection settings.

# pip install capskip
pip install capskip httpx

# Using aiohttp instead? Install it as well. See the note below
# about which HTTP libraries you end up with either way.
pip install capskip aiohttp

The SDK already ships httpx

Worth knowing before you count dependencies. The CapSkip Python package declares requests, httpx and aiofiles as hard runtime dependencies, so installing the SDK installs httpx whether you asked for it or not. If httpx is already your scraping client, the solver adds no new HTTP library to your environment and AsyncCapSkip shares your event loop. If you are on aiohttp, you get httpx alongside it and now have two clients in the same virtualenv. Nothing breaks, but it is the sort of thing a dependency review will ask about.

What changes when there is no browser

In Selenium or Playwright you set the value of the hidden textarea that reCAPTCHA renders, and the page’s own submit handler carries it along. None of that exists here. An HTTP client fetches bytes and nothing evaluates the script tags, so the widget never renders, the textarea is never created, and there is no submit handler to run.

What you have instead is simpler and easier to reason about. The token is just a string, and the site expects it in a POST body under a field name. For reCAPTCHA v2 that field is called g-recaptcha-response, and it is the same name whether the browser filled it or you did. Turnstile uses cf-turnstile-response. GeeTest posts three fields rather than one. Verification happens on the site’s server against Google or Cloudflare, so nobody on the far end can tell which of your processes produced the string.

Step 1: read the sitekey with the client you will submit with

Use one client for the whole flow. That is not tidiness. It is the point: the cookie jar and the connection pool are what make the POST look like it came from the same visitor as the GET. Build a fresh client for the submit and you throw away every session cookie the page set, which is a very common reason a correct token still fails.

# pip install capskip httpx
import re
import httpx

PAGE_URL = "https://example.com/page-with-recaptcha"

# One client for the whole flow. Its cookie jar is what makes
# the POST later look like the same visitor as this GET.
client = httpx.Client(timeout=30, follow_redirects=True)

html = client.get(PAGE_URL).text
match = re.search(r'data-sitekey=["\']([^"\']+)', html)
if not match:
    raise RuntimeError("No data-sitekey in the HTML.")

sitekey = match.group(1)   # this is what the solver needs

If that regex finds nothing, the widget is being injected by JavaScript rather than served in the markup, and an HTTP client will never see it. Open the page in a browser, take the sitekey out of the network tab once, and hardcode it. A sitekey is public and it is stable, so this is not a shortcut you have to feel bad about.

Step 2: solve it on your own machine

The solve is one call against a service listening on your own hardware. Every reCAPTCHA variant is the same method with different keyword arguments: invisible set to 1, enterprise set to 1, or version set to v3 with an action. Turnstile and GeeTest have their own methods with the same shape. The full parameter list is in the CapSkip API documentation.

# CapSkip listens on your machine, so this is a loopback call.
from capskip import CapSkip

solver = CapSkip(host="127.0.0.1", port=8080)
token = solver.recaptcha(sitekey=sitekey, url=PAGE_URL)["code"]

That call blocks while it polls. The SDK does not poll on a flat interval: it starts at 250 milliseconds and backs off towards pollingInterval, which is why an SDK solve usually returns sooner than a loop written by hand against the raw endpoint. The ceiling is recaptchaTimeout, which defaults to 300 seconds.

Step 3: post the token as a form field

Now send it back on the same client, in the same form the browser would have submitted. Include the other fields the form carries, including any hidden CSRF or nonce value you read out of the HTML in step one.

# g-recaptcha-response is an ordinary form field. Same client,
# so the session cookies from the GET go along with it.
reply = client.post(
    PAGE_URL,
    data={
        "username": "demo",
        "g-recaptcha-response": token,
    },
)

print(reply.status_code)

Send the token straight away. A reCAPTCHA token is valid for about two minutes, and a queue, a retry backoff or a sleep between the solve and the submit will burn that budget without you noticing. That lifetime and what it means in practice are covered in the guide to reCAPTCHA token expiration.

Full working example, async

The same three steps with the async client. AsyncCapSkip in Python is a genuine async implementation rather than an alias for the sync one, so it shares your event loop and does not park a thread while it polls.

# pip install capskip httpx
import asyncio
import re
import httpx
from capskip import AsyncCapSkip

PAGE_URL = "https://example.com/page-with-recaptcha"
SITEKEY_RE = re.compile(r'data-sitekey=["\']([^"\']+)')

async def submit_once(client, solver):
    html = (await client.get(PAGE_URL)).text
    match = SITEKEY_RE.search(html)
    if not match:
        raise RuntimeError("No data-sitekey in the HTML.")

    result = await solver.recaptcha(sitekey=match.group(1), url=PAGE_URL)

    reply = await client.post(
        PAGE_URL,
        data={"g-recaptcha-response": result["code"]},
    )
    return reply.status_code

async def main():
    solver = AsyncCapSkip(host="127.0.0.1", port=8080)
    async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
        print(await submit_once(client, solver))

asyncio.run(main())

To run several of these at once, gather them. One solver instance is fine for the whole batch, and one client per identity is usually what you want, because a shared cookie jar means a shared session. The batching side of it is written up in the guide to solving CAPTCHAs in parallel with Python.

The aiohttp version

Everything above holds. Only three names change: the session type, the way you read a body, and the attribute that holds the status code.

# pip install capskip aiohttp
import aiohttp
from capskip import AsyncCapSkip

async def submit_once(session, solver):
    async with session.get(PAGE_URL) as reply:
        html = await reply.text()

    match = SITEKEY_RE.search(html)
    result = await solver.recaptcha(sitekey=match.group(1), url=PAGE_URL)

    async with session.post(
        PAGE_URL,
        data={"g-recaptcha-response": result["code"]},
    ) as submitted:
        return submitted.status   # not status_code

async def main():
    solver = AsyncCapSkip(host="127.0.0.1", port=8080)
    async with aiohttp.ClientSession() as session:
        print(await submit_once(session, solver))

A ClientSession keeps its own cookie jar, so the one-client rule carries over unchanged. aiohttp has no HTTP/2 support, which for this job costs you nothing.

Proxies, and the trap that only bites HTTP clients

If the site checks that the token was generated from the address that submits it, the solve and the submit have to leave from the same proxy. CapSkip takes a proxy per task for reCAPTCHA, Turnstile and GeeTest. Image CAPTCHAs do not accept one, and they do not need one.

# Same exit address for the solve and for the submit.
result = await solver.recaptcha(
    sitekey=sitekey,
    url=PAGE_URL,
    proxy={"type": "HTTP", "uri": "user:[email protected]:3128"},
)

# httpx 0.26 and newer spell this proxy=. Before that it was
# proxies=, which 0.28 removed outright.
async with httpx.AsyncClient(proxy="http://user:[email protected]:3128") as client:
    await client.post(PAGE_URL, data={"g-recaptcha-response": result["code"]})

Here is the part that catches people. CapSkip accepts SOCKS5 and SOCKS5H as proxy types, but aiohttp speaks only HTTP proxies plus HTTPS through a CONNECT tunnel, and httpx needs the socks extra installed before it will do SOCKS at all. Hand the solver a SOCKS5 proxy your client cannot use and the solve succeeds from one address while the submit leaves from another, which looks exactly like a bad token and is not one. Keep both ends on a proxy type they can both speak. Choosing between proxy types is covered in the guide to rotating proxies while solving.

One more aiohttp specific: unlike requests, it ignores the HTTP_PROXY and HTTPS_PROXY environment variables unless you construct the session with trust_env set to True. A proxy you thought was applied simply was not, and nothing told you.

HTTP/2 and what turning it on changes

httpx can speak HTTP/2, but not by default and not without the extra: install httpx with the http2 extra and pass http2 set to True when you build the client. It is worth knowing that this changes how your traffic looks on the wire, because the protocol negotiation and the header ordering differ from HTTP/1.1. That is a fingerprinting question rather than a solving one and it is a separate subject, sketched out in the write-up on TLS fingerprinting in Python.

Running the solver on another machine

Scripts move. A container, a VPS or a scheduled worker is not the machine with the solver on it, and loopback there points at the container, where nothing is listening.

CapSkip has two connection modes. Local binds to 127.0.0.1 and answers that device only. Server binds to your network or public IP, so a container, a VM or a hosted worker reaches the same Windows machine over the API. A static public IP keeps the address stable. It is still your hardware and still unmetered either way, so a busy day costs the same in both modes.

# The SDK reads these three itself, so the same code works
# whether the solver is local or on another machine:
#   CAPSKIP_HOST=192.0.2.10
#   CAPSKIP_PORT=8080
#   CAPSKIP_API_KEY=your-key

solver = AsyncCapSkip()

Turn on key validation once the solver listens on a network address, and give each worker its own key so one can be revoked without touching the rest. Both modes are walked through in the CapSkip setup guide.

Common errors and what they mean

FAQ

The short version

Read the sitekey from the served HTML, solve it locally, then post the token as an ordinary form field on the client that fetched the page. One client for the whole flow, so the cookies survive. Same exit address for the solve and the submit, on a proxy type both ends can actually speak. Post it immediately, because the token is short lived.

The wider Python story is on the Python CAPTCHA solver page. The details of reCAPTCHA v2 live on the reCAPTCHA v2 solver page. The same three calls exist in Node.js, PHP and C# as well, and they are listed on the CAPTCHA solving SDK page.

One last thing worth knowing before you point this at a large crawl. CapSkip is an unlimited captcha solver that runs on hardware you already own, so a crawl that solves fifty thousand challenges costs exactly what a crawl that solves fifty does.