How to Solve CAPTCHAs in AWS Lambda Without Timing Out

An AWS Lambda captcha solve fails in two places, and neither of them is your code. API Gateway stops waiting for the function after 29 seconds, so a reCAPTCHA that takes 40 returns a 504 to the caller while the function is still working. And 127.0.0.1 inside the Lambda sandbox is the sandbox, so a client pointed at loopback finds nothing listening. CapSkip runs on a machine you own, which in this setup is never the one running your function. Fix the address first, then move the solve off the request path.

What you need

• CapSkip running on a Windows machine you control. It is a desktop application and it does not run inside Lambda. The function is the client here, nothing more.
• A Python 3.10 or newer Lambda runtime, with the CapSkip package in the deployment package or in a layer.
• Server mode switched on. Local mode answers on 127.0.0.1 for that device only, which is useless to a function running in AWS. Server mode listens on your network address or public IP so the function can reach it over the same API, and both live under connection settings. A static public IP is recommended, with a firewall rule for the one address AWS will arrive from.
• A way to reach that address from the function. Step 2 covers the two shapes, because a function attached to a VPC behaves differently from one that is not.

Why the API Gateway 29 second timeout decides the design

An AWS Lambda captcha solve has to fit inside three limits. Write them down before you write any code, because together they rule out the obvious design.

So a synchronous API call cannot cover a slow reCAPTCHA. The function has room for it and the gateway in front does not, and the caller sees a 504 while the solve is still running and still being billed.

The workaround people reach for next is worse. Returning early and finishing the solve on a background thread does not work, because after the handler returns, Lambda freezes the execution environment. AWS says it plainly: background processes or callbacks that did not complete when the function ended resume if Lambda reuses the environment. Resume, not continue. Your thread wakes up minutes later, halfway through polling for a CAPTCHA whose token expired long ago, inside an invocation that has nothing to do with it. Nothing errors. The work simply lands in the wrong place. AWS spells out the lifecycle in its execution environment guide.

Step 1: package the SDK and configure the function

Install into a folder and zip it with your handler, or install into a folder named python, zip that, and attach it as a layer. Pin the platform and the interpreter to what the function runs, not to what your laptop runs, or the import fails at cold start with nothing useful in the log. Without those flags pip resolves wheels for your local Python, and a wheel built for a newer interpreter will not load on the runtime.

# pip install capskip
pip install capskip --target package/ \
  --platform manylinux2014_x86_64 --implementation cp \
  --python-version 3.12 --only-binary=:all:

cp lambda_function.py package/
cd package && zip -r ../function.zip . > /dev/null && cd ..

aws lambda update-function-code \
  --function-name solve-captcha --zip-file fileb://function.zip

Then set the timeout and the connection details as configuration rather than in code, so the same package runs against a test solver and a production one.

# Timeout in seconds, and the Server mode address
aws lambda update-function-configuration \
  --function-name solve-captcha \
  --timeout 330 \
  --environment "Variables={CAPSKIP_HOST=203.0.113.10,CAPSKIP_PORT=8080}"

Set the function timeout slightly above the client’s own polling timeout, not below it. Below, and Lambda kills the invocation first, which gives you a bare task timeout in CloudWatch instead of the TimeoutException that would have told you what happened.

Step 2: give the function a NAT gateway and an Elastic IP

This is the step that decides whether anything works, and the answer depends on one setting you may not have thought about as networking.

Lambda documents the first two rows directly: functions have public internet access by default, and attaching one to a VPC limits it to resources inside that VPC until the function’s subnets have a route out. That route is a NAT gateway sitting in a public subnet, described in the Lambda internet access guide. The side effect is the useful part. A NAT gateway holds an Elastic IP, so every solve arrives at your machine from one stable address and your firewall rule can be a single line.

Attach the function to the private subnets, not the public one. This is the trap that produces a hang with a NAT gateway already in place: a function attached to a public subnet has no internet access, whatever the route table says, so the packets simply go nowhere. The same guide repeats that twice.

A NAT gateway is the simplest shape that gives you one stable source address, not the only one. A Site-to-Site VPN or Direct Connect from the same VPC reaches a solver on your own network without exposing its port to the internet at all, and both are worth the setup if the machine is somewhere you would rather not open a port. Either way, keep the solver’s port closed to everything else. Server mode is still your hardware and still unmetered: it only changes where the solver listens so that something other than the same desktop can call it.

Step 3: move the solve off the request path

Given the limits above, an AWS Lambda captcha job belongs on a queue rather than on the request. The handler that answers API Gateway should not be the handler that solves: accept the job, put it on a queue, and answer immediately. A second function reads the queue and does the work with a timeout that suits a CAPTCHA rather than a web request.

import json, os, uuid, boto3

sqs = boto3.client("sqs")
QUEUE_URL = os.environ["QUEUE_URL"]

def lambda_handler(event, context):
    """API Gateway calls this. It never solves anything."""
    body = json.loads(event["body"])
    job_id = str(uuid.uuid4())
    sqs.send_message(
        QueueUrl=QUEUE_URL,
        MessageBody=json.dumps({
            "job_id": job_id,
            "sitekey": body["sitekey"],
            "pageurl": body["pageurl"],
        }),
    )
    return {"statusCode": 202,
            "body": json.dumps({"job_id": job_id})}

The job id is there so the caller has something to ask about later. Design the consumer to finish the work itself rather than to hand a token back, because a token that waits for a second HTTP round trip usually expires on the way.

Use a queue rather than an asynchronous invoke. Lambda retries a failed asynchronous invocation twice by default, and a CAPTCHA solve is the wrong thing to retry blindly: the second attempt starts from a sitekey whose page context has moved on, and you pay for the solve either way. A queue does not remove the retries, it makes them visible and bounded. You get a visibility timeout you control, a redrive policy, and a dead letter queue where a job that keeps failing lands somewhere you can look at it. AWS recommends a maximum receive count of at least five, which leaves room for a throttled retry before the message is parked.

Set the queue’s visibility timeout to at least six times the consumer function’s timeout, which AWS recommends for the same throttling reason. The ordering is not optional: Lambda validates the event source mapping and refuses it if the function timeout is larger than the visibility timeout. With the 330 second function above, that means a visibility timeout near 1980 seconds.

Full working example

The consumer. It builds the client once, outside the handler, so a warm environment reuses it rather than reconnecting on every message.

# pip install capskip
import json, os
from urllib.parse import urlencode
from urllib.request import urlopen
from capskip import (CapSkip, ApiException, NetworkException,
                     TimeoutException, ValidationException)

# Built at cold start and reused while the environment stays warm.
solver = CapSkip(
    host=os.environ["CAPSKIP_HOST"],      # Server mode address
    port=int(os.environ.get("CAPSKIP_PORT", 8080)),
    recaptchaTimeout=300,
)

def lambda_handler(event, context):
    failures = []
    for record in event["Records"]:
        job = json.loads(record["body"])
        try:
            result = solver.recaptcha(
                sitekey=job["sitekey"],
                url=job["pageurl"],
            )
        except NetworkException:
            # No route to the solver. Retry this one message.
            failures.append({"itemIdentifier": record["messageId"]})
            continue
        except (ApiException, TimeoutException, ValidationException) as exc:
            print("giving up on this job:", exc)
            continue

        # Use the token here. It is short lived, so do not park it.
        urlopen(job["pageurl"], data=urlencode(
            {"g-recaptcha-response": result["code"]}).encode())

    # Needs ReportBatchItemFailures on the event source mapping.
    return {"batchItemFailures": failures}

Report the failed message rather than raising. Raising fails the whole batch, and SQS then returns every message in it to the queue, including the ones you already solved, which is the duplicate-solve problem the table below warns about. A partial batch response retries only the record that failed, and it needs the report batch item failures setting on the event source mapping to be honoured.

Retry a NetworkException and swallow the other three. No route to the solver means the job can succeed later, while an unsolvable CAPTCHA, a timeout or a bad parameter will fail the same way on every attempt and retrying it just spends the same time again. All four SDK exceptions derive from CapSkipError if you would rather catch one thing.

That submit is the point of the whole design: do the thing the token is for inside the same invocation. A reCAPTCHA token is good for about two minutes, so writing it to a database for a later step to collect usually means collecting something that has already expired. The details of that window are in the reCAPTCHA v2 solver guide, and the raw endpoints behind every SDK call are documented on the API reference.

Common errors and what they mean

FAQ

The short version

An AWS Lambda captcha solve needs three decisions and they are all made before you write the handler. Switch CapSkip to Server mode, because loopback in the Lambda sandbox reaches nothing. Attach the function to a VPC and route it through a NAT gateway so your firewall has one Elastic IP to allow. Then stop solving on the request path: API Gateway gives you 29 seconds, a slow reCAPTCHA needs more, and returning early does not help because the environment freezes the moment your handler does. Queue the job, solve it in a consumer whose timeout sits above the client’s, and use the token in the same invocation that earned it.

Every method the Python package exposes, with the options each one takes, is listed on the Python CAPTCHA solver page.

One last point about the economics, because it is what makes the queue design comfortable. A discarded job costs you the Lambda milliseconds and nothing else: the captcha bypass work happens on hardware you already paid for, so retrying a job or throwing an expired token away never shows up on an invoice from anyone.