[Feature]: Crawl4aiDockerClient has no public option for the API token the 0.9 server requires #2474

crawl4ai version

0.9.5 (also on develop; since 0.9.0)

Expected Behavior

The SDK client can talk to a 0.9 server, which requires CRAWL4AI_API_TOKEN on every request.

Current Behavior

Crawl4aiDockerClient has no token option, and every request returns:

401 Authentication required

client.authenticate(email) does not help: it posts only {"email": ...} to /token, but since 0.9.0 /token also requires api_token in the body, so it returns 401 (or 403 when no token is configured).

The only working way is to set a private attribute, as deploy/docker/README.md does today:

client._token = os.environ["CRAWL4AI_API_TOKEN"]
# or
client._http_client.headers["Authorization"] = "Bearer <token>"

Is this reproducible?

Yes

Code snippets

from crawl4ai.docker_client import Crawl4aiDockerClient

async with Crawl4aiDockerClient(base_url="http://localhost:11235") as client:
    await client.crawl(["https://example.com"])
# RequestError: Server error 401: Authentication required

Proposed fix

Add api_token: Optional[str] = None to Crawl4aiDockerClient.__init__ and send it as Authorization: Bearer <token> on every request. With the private _token workaround the header is set only inside crawl(), so get_schema() still returns 401. I'll open a PR.

The broken authenticate() is left for a separate change.

OS

macOS 26.6 (platform-independent)

Python version

3.13