To solve CAPTCHAs in pytest, build one CapSkip client per session in conftest.py and solve inside each test, because a token works once. Skip the CAPTCHA tests when CapSkip is not running. Then give the SDK its own timeouts. On Windows, pytest-timeout can only use its thread method, and when that fires it ends the whole run. The Python SDK sets no HTTP timeout, so a silent CapSkip would stall a test with no error. CapSkip runs on your own Windows PC, so a suite that solves hundreds of times a night has no per-solve cost. This pytest CAPTCHA setup covers each of those traps, plus parallel runs with xdist and async tests.<\/p>\n
The measurements below come from a mock CapSkip and a mock login page on the same PC, used with the real pytest and the real SDK. The pytest behaviour and the SDK behaviour hold for your own site too. Solve times, thread limits and the accept and reject rules of the login page are the mock’s own.<\/p>\n
Put everything in one project folder: conftest.py, tokens.py and the test files. Use a virtual environment. Every command below runs the virtual environment’s own Python by its path, so you do not need to activate it. Start pytest as python -m pytest. That form adds the current folder to the import path. If you move conftest.py into a tests folder, keep tokens.py in the project folder. Then pytest.exe fails with ModuleNotFoundError: No module named ‘tokens’, and python -m pytest works.<\/p>\n
# Windows PowerShell, in your project folder\npython -m venv .venv\n.venv\\Scripts\\python -m pip install pytest "capskip==1.3.0" requests<\/pre>\n<\/div>\nStep 2: one CapSkip client for the whole run<\/h2>\n
Save this as conftest.py. It reads the CapSkip settings from the environment, because the SDK does not read them on its own, and the defaults make it run with no setup. CAPSKIP_API_KEY only matters if you switched on API key validation in CapSkip. The fixture builds one client for the whole session. It also finds out whether CapSkip is running.<\/p>\n
\n# conftest.py (next to your tests): pip install pytest capskip requests\nimport os\nimport socket\n\nimport pytest\nfrom capskip import CapSkip\n\n# The SDK reads no environment variables, so the fixture reads them.\nHOST = os.environ.get("CAPSKIP_HOST", "127.0.0.1")\nPORT = int(os.environ.get("CAPSKIP_PORT", "8080"))\nAPI_KEY = os.environ.get("CAPSKIP_API_KEY", "capskip")\n\n\n@pytest.fixture(scope="session")\ndef capskip_client():\n # CapSkip has no ping endpoint, so try a plain TCP connect. Use the IP,\n # not "localhost": a refused connect takes 2 s, or 4 s for localhost.\n try:\n socket.create_connection((HOST, PORT), timeout=3).close()\n except OSError as exc:\n pytest.skip(f"CapSkip is not running on {HOST}:{PORT}: {exc}")\n # One client for the whole run. A solve gives up after 90 s of polling.\n return CapSkip(apiKey=API_KEY, host=HOST, port=PORT, recaptchaTimeout=90)<\/pre>\n<\/div>\nCapSkip has no ping endpoint, so a plain TCP connect asks whether anything is listening, and it sends no solve job. If nothing is, the fixture skips instead of failing. With CapSkip closed, a refused connect to 127.0.0.1 took 2.05 seconds on Windows, and about 4.1 seconds to the name localhost. The skip is cached, so a whole session pays that cost once: four CAPTCHA tests in two files took 2.05 seconds in total.<\/p>\n
Use the IP. The name localhost has a second cost even when CapSkip is running. Against an IPv4-only listener, two login tests took 10.3 seconds with localhost and 3.75 seconds with 127.0.0.1. That is about 2 seconds once for the connection check, and 1 to 2.5 seconds more for each solve.<\/p>\n
Step 3: a helper that solves once, and retries once<\/h2>\n
Save this as tokens.py. Some solves fail, and a fresh attempt can work. The SDK raises an ApiException whose message is ERROR_CAPTCHA_UNSOLVABLE, unless you set an Error Replacement text in CapSkip. Then the message carries that text, and the check below never matches. The helper retries only that failure. A submit that CapSkip refuses, such as ERROR_GOOGLEKEY or ERROR_PAGEURL, raises at once. A sitekey that Google refuses reads as UNSOLVABLE too, so it is tried twice before it fails. See ERROR_CAPTCHA_UNSOLVABLE<\/a> for what the error means.<\/p>\n
\n# tokens.py: the helper your tests (and your app) call\nfrom capskip import ApiException\n\n\ndef fresh_token(solver, sitekey, url, attempts=2):\n """Solve a reCAPTCHA v2 and return the token string."""\n for attempt in range(attempts):\n try:\n return solver.recaptcha(sitekey=sitekey, url=url)["code"]\n except ApiException as exc:\n # UNSOLVABLE is how CapSkip reports a solve that failed this\n # time. A refused submit (ERROR_GOOGLEKEY, ERROR_PAGEURL) will\n # fail again, so it raises at once.\n last = attempt == attempts - 1\n if last or "ERROR_CAPTCHA_UNSOLVABLE" not in str(exc):\n raise<\/pre>\n<\/div>\nStep 4: a fixture per test, and a marker<\/h2>\n
Add the next block to the end of conftest.py. It is a factory, not a token. Every call to it is a new solve, and each test asks for its own, because a reCAPTCHA token is accepted once. Python allows imports in the middle of a file, so the import line can stay where the block puts it.<\/p>\n
\nfrom tokens import fresh_token\n\n\n@pytest.fixture\ndef solve_recaptcha(capskip_client):\n # A factory, not a token: every call is a new solve, because a\n # reCAPTCHA token is accepted once. Function scope, so each test\n # asks for its own.\n def solve(sitekey, url):\n return fresh_token(capskip_client, sitekey, url)\n\n return solve<\/pre>\n<\/div>\nThen add the marker hook below to conftest.py. It registers a marker called captcha, so that you can run the fast tests without CapSkip and the live ones with it. With the strict markers option on, which Step 6 switches on, any marker you did not register is an error. Registering captcha keeps the real one working and turns a misspelling into an error.<\/p>\n
\ndef pytest_configure(config):\n # Tell pytest the marker exists, so --strict-markers accepts it and a\n # misspelling such as "captcah" is an error instead of a silent mark.\n config.addinivalue_line(\n "markers", "captcha: needs a running CapSkip and solves a real CAPTCHA"\n )<\/pre>\n<\/div>\nStep 5: write the login tests<\/h2>\n
Save the next two blocks as one file, test_login.py. This first block is yours, not CapSkip’s. It opens the login page, reads the CSRF token and the sitekey, and posts the form. requests sends no Referer on a form post. A strict site such as Django over HTTPS rejects a post without one, and the mock did the same with a 403. So the helper sets it. Adjust the two patterns and the field names to your markup, and put the email and password of a test account into post_form.<\/p>\n
\n# test_login.py: set LOGIN_URL to your own login page\nimport os\nimport re\n\nimport pytest\nimport requests\n\nLOGIN_URL = os.environ.get("LOGIN_URL", "https:\/\/example.com\/login")\n\n\ndef open_form(session):\n html = session.get(LOGIN_URL, timeout=15).text\n csrf = re.search(r'name="csrfmiddlewaretoken" value="([^"]+)"', html)\n sitekey = re.search(r'data-sitekey="([^"]+)"', html)\n return csrf.group(1), sitekey.group(1)\n\n\ndef post_form(session, csrf, token, password="secret"):\n # requests sends no Referer, and a Django site refuses an HTTPS post\n # that has none, so set it by hand.\n return session.post(LOGIN_URL, timeout=15, headers={"Referer": LOGIN_URL}, data={\n "csrfmiddlewaretoken": csrf,\n "email": "user@example.com",\n "password": password,\n "g-recaptcha-response": token,\n })<\/pre>\n<\/div>\nThe tests solve last and post at once, because the token expires after about two minutes. The guide to reCAPTCHA token expiration<\/a> explains that side of it. Change Welcome, the 403 and the error text to what your own site shows. A fixed username collides on the second nightly run, so use a new one each time if your form creates accounts.<\/p>\n
\n@pytest.mark.captcha\ndef test_login(solve_recaptcha):\n session = requests.Session()\n csrf, sitekey = open_form(session)\n token = solve_recaptcha(sitekey, LOGIN_URL) # solve last, post at once\n resp = post_form(session, csrf, token)\n assert resp.status_code == 200 and "Welcome" in resp.text\n\n\n@pytest.mark.captcha\ndef test_wrong_password_is_refused(solve_recaptcha):\n session = requests.Session()\n csrf, sitekey = open_form(session)\n token = solve_recaptcha(sitekey, LOGIN_URL) # a new solve, not test 1's\n resp = post_form(session, csrf, token, password="wrong")\n assert resp.status_code == 403\n assert "Invalid email or password" in resp.text<\/pre>\n<\/div>\nStep 6: run the fast tests, then the live ones<\/h2>\n
The first line points the tests at your login page. The second makes a misspelled marker an error. It sets an environment variable that pytest reads for extra options, and it lasts for the rest of that PowerShell session. The third line runs everything except the CAPTCHA tests. The fourth runs only the live ones. The skip switch prints why a test was skipped, and the durations switch lists the slowest tests.<\/p>\n
\n# Windows PowerShell: your login page, then make a misspelled marker an error\n$env:LOGIN_URL = "https:\/\/staging.example.com\/login"\n$env:PYTEST_ADDOPTS = "--strict-markers"\n.venv\\Scripts\\python -m pytest -m "not captcha" # fast suite, no CapSkip\n.venv\\Scripts\\python -m pytest -m captcha -rs -v --durations=5 # live tests<\/pre>\n<\/div>\nAt this point the fast run finds nothing to run, and pytest exits with code 5. It has tests once you save test_unit.py from the unit tests section further down. Then it needs nothing from the solver: its four tests passed in 0.02 seconds with CapSkip closed. The live run skipped with a line like this, pointing at the test and not at conftest.py:<\/p>\n
\n
- SKIPPED [1] test_login.py:29: CapSkip is not running on 127.0.0.1:8080: [WinError 10061] No connection could be made because the target machine actively refused it<\/li>\n<\/ul>\n
A skipped suite exits with code 0, and pytest 9.1.1 has no switch that turns a skip into a failure. If your build must have CapSkip, write a junit file with the junitxml option and fail the build when its skipped count is above zero. Typing captcah instead of captcha in a test, with the strict option on, stops collection with exit code 2 and the message ‘captcah’ not found in `markers` configuration option. Without the strict option the typo would only be a warning, and that test would run in the fast suite.<\/p>\n
Solve inside every test<\/h2>\n
Do not solve once in a fixture and share the token. In testing, a module-scoped fixture solved once and two tests posted the same token. The first was accepted. The second failed with the mock’s reCAPTCHA failed: token already used. A real site words it differently. Which test loses depends on the order, so the failure moves around and looks flaky. It is worse under xdist, as the parallel section shows. Here is the mistake in full, so you can recognise it. Delete the file after you have seen it fail, or it will make later runs fail or pass at random.<\/p>\n
\n# test_shared_token.py: the mistake, so you can see it fail\nimport pytest\nimport requests\n\nfrom test_login import LOGIN_URL, open_form, post_form\n\n\n@pytest.fixture(scope="module")\ndef shared_token(capskip_client):\n csrf, sitekey = open_form(requests.Session())\n return capskip_client.recaptcha(sitekey=sitekey, url=LOGIN_URL)["code"]\n\n\n@pytest.mark.captcha\n@pytest.mark.parametrize("n", [1, 2])\ndef test_each_login_reuses_one_token(shared_token, n):\n session = requests.Session()\n csrf, sitekey = open_form(session)\n resp = post_form(session, csrf, shared_token)\n assert resp.status_code == 200, resp.text # n=2 fails: used already<\/pre>\n<\/div>\nOne solve per test costs a few seconds each. A test that does not need the CAPTCHA itself should not ask for the fixture. The fake solver further down covers the logic around it.<\/p>\n
Make a hung solve fail, not abort the run<\/h2>\n
This is the part that is different on Windows. The pytest-timeout plugin has two methods. The signal method needs SIGALRM, which Windows does not have, so it is not available there. Asking for it ended with INTERNALERROR and AttributeError: module ‘signal’ has no attribute ‘SIGALRM’. The thread method is the default, and it is the only one that works. Here is what it did when a test hung, with a limit of 10 seconds and four tests in the run:<\/p>\n
\n
- It printed the main thread’s stack, about 100 lines, and ended the process with exit code 1 at 12.5 seconds.<\/li>\n
- The hung test got no PASSED or FAILED line.<\/li>\n
- No summary was printed and no junit file was written.<\/li>\n
- The two tests after it never ran.<\/li>\n<\/ul>\n
With xdist the damage is smaller. Only the worker running that test dies. The test is reported as FAILED with worker gw1 crashed while running it, a replacement worker starts, and the run finishes with a summary and exit code 1.<\/p>\n
So the thread timeout is a last resort, not a test timeout. You want the SDK to fail first, with an ordinary exception, so the test fails and the run goes on. The Python SDK 1.3.0 makes that hard. It calls requests with no timeout, so a CapSkip that accepts a call and never answers hangs it. Without any change, a silent submit was still blocked when we stopped it at 45 seconds. Its recaptchaTimeout is only checked between polls, and a hung poll is treated as not ready.<\/p>\n
The fix is to swap the two request functions the SDK uses for ones with a timeout. Add the next block to conftest.py. It reaches into the SDK, so pin capskip to version 1.3.0. The hasattr line makes it do nothing if a later version stops using requests. It does not cover the download of an image CAPTCHA given as a URL, because the SDK does that with its own call.<\/p>\n
\nimport functools\nimport types\n\nimport capskip.api\nimport requests\n\n# capskip 1.3.0 sets no HTTP timeout, so a CapSkip that accepts a call\n# and never answers would hang the test. Give every SDK call 5 s to\n# connect and 30 s to answer. This swaps a name inside the SDK, so pin\n# capskip==1.3.0 and do nothing if the SDK stops using requests.\nif hasattr(capskip.api, "requests"):\n capskip.api.requests = types.SimpleNamespace(\n RequestException=requests.RequestException,\n post=functools.partial(requests.post, timeout=(5, 30)),\n get=functools.partial(requests.get, timeout=(5, 30)),\n )<\/pre>\n<\/div>\nWith the patch, a CapSkip that never answered the submit raised NetworkException with Read timed out. (read timeout=30) at 30.0 seconds. The pytest test failed at 30.8 seconds, and the run stayed intact. A silent poll is different. The SDK swallows it as not ready, and the default recaptchaTimeout of 300 seconds ended with TimeoutException at 333.0 seconds. That is why Step 2 passes recaptchaTimeout=90. With 90, the test failed at 92.6 seconds.<\/p>\n
The deadline can be overshot by up to about 35 seconds. It is checked between polls, a poll can take its full 30 seconds, and the SDK waits up to 5 seconds before it checks. With a limit of 45 the failure came at 60.8 seconds, and with 20 at 30.3 seconds. So the worst case for one attempt with these numbers is about 125 seconds: 90 for the polling, plus one stuck poll. The helper from Step 3 makes up to two attempts, so the worst case is about 250 seconds. Set the pytest-timeout limit above that. The next block installs the plugin and sets 300 seconds. The environment variable replaces the one from Step 6, so it repeats the strict option. You can put the same two options in a pytest ini file as timeout and timeout_method instead. The variable is just easier to paste.<\/p>\n
\n# Windows PowerShell: pytest-timeout has only the thread method here.\n# 300 s is above two attempts at the slowest SDK failure (about 125 s each),\n# so the SDK fails first.\n.venv\\Scripts\\python -m pip install pytest-timeout\n$env:PYTEST_ADDOPTS = "--strict-markers --timeout=300 --timeout-method=thread"<\/pre>\n<\/div>\nRun the tests in parallel with xdist<\/h2>\n
Solving is slow and mostly waiting, so xdist helps. Each xdist worker is a separate process. A session fixture therefore runs once per worker, and every worker builds its own client. With four workers and eight tests, the run showed four process IDs and four client objects. Without xdist there is one of each. The test writes workers-0.log to workers-7.log, and Get-Content workers-*.log shows them. One client per worker is fine, because the client keeps nothing that workers need to share.<\/p>\n
\n# test_workers.py: shows one client per worker process\nimport os\nfrom pathlib import Path\n\nimport pytest\n\n\n@pytest.mark.captcha\n@pytest.mark.parametrize("n", range(8))\ndef test_which_client(capskip_client, worker_id, n):\n line = f"{worker_id} pid={os.getpid()} client={id(capskip_client)}"\n Path(f"workers-{n}.log").write_text(line)<\/pre>\n<\/div>\nPick the number of workers yourself. The auto setting means the CPU count, not what CapSkip can solve at once. It gave 24 workers on the test PC. Four unit tests took about 4 seconds with it and about 1.1 seconds with four workers, because of the worker start-up cost. CapSkip’s own limit is the reCAPTCHA Max. Threads setting, which defaults to 10. This table is from 24 logins against a mock with that limit and a 5 second solve.<\/p>\n
\n\n
\n \nWorkers<\/th>\n Run time<\/th>\n Solving at once<\/th>\n<\/tr>\n<\/thead>\n \n 1<\/td>\n 189.6 s<\/td>\n 1<\/td>\n<\/tr>\n \n 4<\/td>\n 49.1 s<\/td>\n 4<\/td>\n<\/tr>\n \n 10<\/td>\n 26.1 s<\/td>\n 10<\/td>\n<\/tr>\n \n 12<\/td>\n 24.1 s<\/td>\n 10<\/td>\n<\/tr>\n \n 24<\/td>\n 23.1 s<\/td>\n 10<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Past 10 workers the gain is small: 26.1 seconds with 10 workers and 23.1 with 24. The extra jobs wait for a free thread inside CapSkip, and a job that waits past the Wait Timeout fails. In a scaled-down test, eight tests ran at once against a limit of 2, a wait limit of 6 seconds and a 4 second solve. Four jobs failed with ERROR_CAPTCHA_UNSOLVABLE. The retry in tokens.py queued them again, and all eight tests passed in 22.3 seconds. That is the second reason for the retry.<\/p>\n
Each test took about 7.9 seconds for a 5 second solve. The SDK polls at 0, 0.25, 0.75, 1.75, 3.75 and 7.75 seconds, so it notices a finished solve at the next poll. The xdist install and the command are below.<\/p>\n
\n# Each xdist worker is its own process, so each builds its own client.\n# Pick the number yourself: -n auto means CPU count, not solver capacity.\n.venv\\Scripts\\python -m pip install pytest-xdist\n.venv\\Scripts\\python -m pytest -m captcha -n 4 -v<\/pre>\n<\/div>\nOne more trap. Under xdist, the shared-token mistake from earlier can pass. The module-scoped token is solved once per worker, so two tests collide only when they land on the same worker. A test that passes in some runs and fails in others is a clue to look for a shared token.<\/p>\n
Async tests with AsyncCapSkip<\/h2>\n
The Python SDK has a real async client, AsyncCapSkip. It is built on httpx. Add this fixture to conftest.py. It asks for capskip_client, so the skip check still applies. The async client holds no event loop, so a plain fixture is enough and the tests run on the default function-scoped loops.<\/p>\n
\nfrom capskip import AsyncCapSkip\n\n\n@pytest.fixture(scope="session")\ndef async_capskip_client(capskip_client):\n # Asking for capskip_client keeps the skip-when-closed check. The\n # async client holds no event loop, so a plain fixture is enough.\n return AsyncCapSkip(apiKey=API_KEY, host=HOST, port=PORT, recaptchaTimeout=90)<\/pre>\n<\/div>\npytest-asyncio 1.4.0 runs in strict mode, and pytest 9 fails an async test that lacks the marker with Failed: async def functions are not natively supported. Save the next block as test_async.py. Change YOUR_SITEKEY to the sitekey of the page.<\/p>\n
\n# test_async.py: .venv\\Scripts\\python -m pip install pytest-asyncio\nimport asyncio\n\nimport pytest\n\nfrom test_login import LOGIN_URL\n\n\n@pytest.mark.captcha\n@pytest.mark.asyncio # needed: pytest-asyncio runs in strict mode\nasync def test_five_solves_at_once(async_capskip_client):\n solves = [\n async_capskip_client.recaptcha(sitekey="YOUR_SITEKEY", url=LOGIN_URL)\n for _ in range(5)\n ]\n results = await asyncio.gather(*solves)\n assert len({r["code"] for r in results}) == 5 # five different tokens<\/pre>\n<\/div>\n