[Bug]: deep crawl configs from dump() are rejected by the server (logger: null) and DFS strategies dump empty #2450

crawl4ai version

0.9.5 (also on develop)

Expected Behavior

A deep crawl config produced by CrawlerRunConfig.dump(), as Crawl4aiDockerClient sends it, is accepted by the server when CRAWL4AI_ALLOW_DEEP_CRAWL=true.

Current Behavior

Two problems in to_serializable_dict() stop it:

1. logger: null. Objects that cannot be deserialized (a Logger, a callable) serialize to None, but the field is still written. Every deep crawl strategy dumps as {"max_depth": ..., "logger": null}, and the server rejects logger on deep crawl strategies from an untrusted request even when it is null:

   UntrustedConfigError: field 'logger' is not permitted on BFSDeepCrawlStrategy from an untrusted request
   

2. DFSDeepCrawlStrategy dumps with no parameters. Its __init__ only forwards *args, **kwargs to BFSDeepCrawlStrategy, so there are no named parameters to read back: DFSDeepCrawlStrategy(max_depth=3, max_pages=7) dumps as {"type": "DFSDeepCrawlStrategy", "params": {}}, and loading it fails with TypeError: ... missing 1 required positional argument: 'max_depth'. The logger error currently hides this one.

Is this reproducible?

Yes

Inputs Causing the Bug

Any CrawlerRunConfig with a deep crawl strategy, loaded with Provenance.UNTRUSTED (the server path).

Steps to Reproduce

Run the snippet below; it reproduces the server's check in-process.

Code snippets

import json
import os

os.environ["CRAWL4AI_ALLOW_DEEP_CRAWL"] = "true"
from crawl4ai import CrawlerRunConfig
from crawl4ai.async_configs import Provenance, to_serializable_dict
from crawl4ai.deep_crawling import BFSDeepCrawlStrategy, DFSDeepCrawlStrategy

cfg = CrawlerRunConfig(deep_crawl_strategy=BFSDeepCrawlStrategy(max_depth=1))
print(cfg.dump()["params"]["deep_crawl_strategy"]["params"])  # {'max_depth': 1, 'logger': None}
CrawlerRunConfig.load(json.loads(json.dumps(cfg.dump())), provenance=Provenance.UNTRUSTED)
# UntrustedConfigError: field 'logger' is not permitted on BFSDeepCrawlStrategy from an untrusted request

print(to_serializable_dict(DFSDeepCrawlStrategy(max_depth=3, max_pages=7)))
# {'type': 'DFSDeepCrawlStrategy', 'params': {}}

OS

Windows 11 (platform-independent)

Python version

3.13

Browser

Not involved

Error logs & Screenshots (if applicable)

See above.

The logger part was found by @SohamKukreti in the review of #2425; the DFS part showed up once it was fixed. I'll open a PR.