[Bug]: FreshnessScorer defaults to current_year=2024, so this year's URLs score like undated ones #2426

crawl4ai version

0.9.5 (also on main and develop)

Expected Behavior

As the docstring says, "more recent years score higher": with the default settings a URL dated this year scores highest.

Current Behavior

current_year defaults to a hard-coded 2024, and _extract_year discards any year later than current_year as invalid. So today, with the default:

• a /2026/ or /2025/ URL scores 0.5, the same as a URL with no date
• a /2024/ URL scores 1.0

BestFirstCrawlingStrategy with a default FreshnessScorer therefore ranks 2024 content above current content. The docs use it with the default (FreshnessScorer(weight=0.8)).

Is this reproducible?

Yes

Inputs Causing the Bug

Any FreshnessScorer() without current_year.

Steps to Reproduce

Run the snippet below.

Code snippets

from crawl4ai.deep_crawling.scorers import FreshnessScorer

scorer = FreshnessScorer()
for url in [
    "https://example.com/blog/2026/10/01/post",
    "https://example.com/blog/2024/01/01/post",
    "https://example.com/about",
]:
    print(url, scorer.score(url))
# 2026 -> 0.5, 2024 -> 1.0, undated -> 0.5

OS

Windows 11 (platform-independent)

Python version

3.13

Browser

Not involved

Error logs & Screenshots (if applicable)

None; no error is raised.

Fix: default current_year to None and use date.today().year; an explicit current_year keeps working as before. I'll open a PR against develop referencing this issue.

Investigated with AI assistance; I ran the reproduction and checked the output myself.