Repository navigation
Tags: browser-use/browser-use
Tags
Fix OpenAI Cloud navigation and scaled screenshots (#6089) Cloud can briefly report an empty URL while a tab navigates. OpenAI 3.31 rejects that URL during its post-action browser-state check, which stops the tool runner before it returns the action result. Represent an uncommitted tab as about:blank so OpenAI accepts the state while navigation completes. The driver still reports committed URLs unchanged. Also resize screenshots to CSS viewport pixels so coordinate actions and crops remain aligned on scaled displays. Prepare Browser Use 0.13.13. The regression test exercises the real OpenAI toolset and real Chromium with one empty provider URL. It fails against the original implementation and passes with this fix. A second real-browser regression proves the screenshot mismatch at double display scale and verifies the corrected pointer click. All eleven driver tests and pre-commit checks pass. The first five-task Cloud pilot passed CDC and PlayStation, then stopped on Southwest, EPA, and Good Food. A separate diagnostic retry traced the Southwest failure to OpenAI's URL validation. Each original task stopped its Cloud browser. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes browser-state reporting and screenshot dimensions used by vision/coordinate tools; low blast radius but affects OpenAI tool reliability on Cloud and scaled displays. > > **Overview** > Fixes two OpenAI browser-tool integration issues in **`BrowserUse`**: transient invalid tab URLs during navigation and screenshot/pointer misalignment on high-DPI viewports. Release **0.13.13**. > > **Tab state:** When CDP reports an empty tab URL (common while Cloud navigation is in flight), **`state()`** now exposes **`about:blank`** instead of a blank string so OpenAI’s post-action URL validation does not abort the tool run. Committed URLs are unchanged when CDP provides them. > > **Screenshots:** **`screenshot()`** reads the CSS visual viewport via **`Page.getLayoutMetrics`** and downscales the PNG to **`clientWidth` × `clientHeight`** when the capture is larger (e.g. **`deviceScaleFactor: 2`**), keeping **`click_at`**, **`zoom`**, and crops in the same CSS pixel space as **`_bounds`**. > > New CI tests cover empty provider URLs through the real **`AsyncBrowserToolset`** path and a double-scale screenshot + coordinate click regression. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 40166d3. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes OpenAI Cloud tool failures from empty tab URLs during navigation and misaligned screenshots on scaled displays. **Bug Fixes** - Uncommitted tabs now report `about:blank` so OpenAI accepts the browser state while navigation finishes; committed URLs are still reported unchanged. - Screenshots are resized to CSS viewport pixels so coordinate actions and crops stay aligned on scaled displays. - Bumps `browser-use` to 0.13.13. Two real-browser regression tests cover the empty-URL case and the screenshot/pointer alignment at double display scale. <sup>Written for commit 40166d3. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/6089?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <a href="https://www.cubic.dev/action/auto-fix/pr/browser-use/browser-use/6089?returnTo=https%3A%2F%2Fgithub.com%2Fbrowser-use%2Fbrowser-use%2Fpull%2F6089&source=description" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"><img alt="Turn on auto-fix" src="https://www.cubic.dev/buttons/turn-on-auto-fix-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
Add OpenAI browser tools backed by Browser Use Actor (#6086) OpenAI browser helpers require a provider driver before they can run Browser Use actions. Add `browser_use.integrations.openai.BrowserUse`, backed by the existing Actor API, for local and Cloud browsers. The driver implements the 18 default tools and six optional actions. It preserves borrowed browser ownership, rejects stale element references, waits for tab closure, and checks Cloud cleanup. Uploads, downloads, scoped reads, and diagnostics remain unsupported. The optional `openai-tools` extra installs OpenAI 3.31+. URL and timeout conversion preserves existing model-wrapper settings across OpenAI 2.x and 3.x. Update the LangChain example dependency for OpenAI 3.x resolution. Prepare version 0.13.12. Validation: 35 integration/model tests passed, two credential-dependent tests skipped. OpenAI 2.26 compatibility: 15 passed, one skipped. Full dependency resolution, pre-commit, type checks, and wheel content checks passed. Live Cloud navigation and element-reference click tests passed in run38824343560. Both received score1.0 and confirmed browser cleanup. All GitHub checks passed. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > <sup>[Cursor Bugbot](https://cursor.com/bugbot) is generating a summary for commit ba5b05e. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
chore: pin release dependencies for 0.13.10 (#5670) ## Summary - release Browser Use 0.13.10 with exact pins for browser-harness 0.1.13, Pydantic 2.13.5, pydantic-settings 2.15.0, MCP 2.1.1, and pypdf 6.16.2 - migrate both MCP servers and the MCP client/controller to the MCP 2 low-level server API - pin Hatchling 1.32.0 - make pydantic-settings an explicit runtime dependency instead of relying on MCP 1 to provide it transitively The publish-workflow uv pin is split into #5669 because repository policy requires workflow-guardians approval for `.github/workflows/**`. This PR replaces #5667 with identical source tree changes but clean history so the workflow-only rule is scoped correctly. ## Security - pypdf 6.16.2 resolves all three open Dependabot alerts on main (GHSA-23w6-3w8w-8484, GHSA-763m-79hh-57f2, GHSA-jp53-mhqp-8xcg) - all external direct, optional, dev, and build dependencies in pyproject.toml are exact-pinned - all newly selected PyPI artifacts checked have digital provenance attestations - isolated installed-runtime `pip-audit`: no known vulnerabilities ## Verification - all GitHub test shards passed on the identical code tree in #5667 - code style, type checker, CodeQL, GitGuardian, and Cubic review passed - clean wheel installs and CLI smoke passed on macOS, Linux, Windows, and uvx - real stdio MCP initialization/list-tools passed for both `browser-use --mcp` and `browser-use --cli-mcp` - 204 focused PDF/save/filesystem tests passed - wheel and sdist build passed A local live OpenAI agent probe could navigate to example.com, but the configured local OpenAI credential returns 401. GitHub model adapter tests passed. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Pins release dependencies for 0.13.10 and migrates the MCP servers and client to MCP 2's low-level request-handler API. - Exact pins: `browser-harness==0.1.13`, `pydantic==2.13.5`, `pydantic-settings==2.15.0`, `mcp==2.1.1`, `pypdf==6.16.2`, `hatchling==1.32.0`. - `pydantic-settings` is now an explicit runtime dependency instead of coming transitively via MCP 1. - The MCP server now reports the real package version instead of the hardcoded `0.1.0`. **Migration** - Request handlers registered via `add_request_handler` replace the `list_tools`/`call_tool` decorators. - Tool schemas and results use MCP 2 field names: `input_schema`, `is_error`, `read_only_hint`. - Tool failures and unknown tool calls now return `is_error=True` results instead of plain error text. **Security** - `pypdf==6.16.2` fixes three Dependabot alerts (GHSA-23w6-3w8w-8484, GHSA-763m-79hh-57f2, GHSA-jp53-mhqp-8xcg). <sup>Written for commit 743f630. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/5670?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
chore: release 0.13.9 with Browser Harness 0.1.12 (#5666) ## Summary - bump `browser-use` to 0.13.9 - pin `browser-harness==0.1.12` - sync the bundled Browser Use skill from the immutable Browser Harness v0.1.12 tag - allow the documented iTerm app name in codespell This ships the persistent local approval flow from browser-harness v0.1.12, including the `mac-approve` guidance. ## Validation - `uv run pre-commit run --all-files --show-diff-on-failure` - isolated full suite: 1,136 passed, 34 skipped - `uv build --wheel` - clean wheel install verified browser-use 0.13.9, browser-harness 0.1.12, and packaged `mac-approve` skill guidance - live Anthropic provider test passed (`claude-sonnet-4-6`) - end-to-end agent smoke passed: opened Hacker News and extracted the top three stories with points and comment counts in two steps, zero errors <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Ships `browser-use` 0.13.9 with `browser-harness` 0.1.12. This replaces the standalone `mac-approve` run with a persistent approval flow: keep the original browser command running and call `mac-approve` with the matching `BU_NAME`. - Syncs the bundled Browser Use skill from the Browser Harness v0.1.12 tag. - Documents `BH_TAB_MARKER=0` to leave page titles unchanged. - Adds guidance to avoid slow per-character typing for long text. - Allows the documented `iterm` app name in codespell. <sup>Written for commit 0e0983e. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/5666?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
Release 0.13.8 with Browser Harness 0.1.9 (#5481) ## Summary - bump Browser Use from 0.13.7 to 0.13.8 - upgrade `browser-harness` from 0.1.8 to 0.1.9 - align Pillow at 12.3.0 and `browser-use-core` at 0.13.3 so the dependency set is resolvable - sync both packaged Browser Use skill copies from the immutable Browser Harness `v0.1.9` tag Prerequisites are published and verified on PyPI: - `browser-harness==0.1.9` - `browser-use-core==0.13.3` (all five platform wheels) ## Validation - `uv sync --dev --all-extras --python 3.11 --refresh-package browser-use-core` - `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` - 6 focused CLI/skill-install tests passed - tagged and current-main Browser Harness skill sync checks passed - `uv build --wheel` - strict clean-venv install of the built `browser-use[core]` wheel - verified resolved versions: Browser Use 0.13.8, Harness 0.1.9, Core 0.13.3, Pillow 12.3.0 - verified `browser-use --help`, `browser-use auth --help`, `browser-use skill show`, all CLI aliases, and `browser-use-terminal --version` Publishing the Browser Use GitHub release is intentionally left for manual execution after this PR merges. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Releases `browser-use` 0.13.8 to align with `browser-harness` 0.1.9 and a resolvable dependency set (`browser-use-core` 0.13.3, `pillow` 12.3.0). Updates packaged skill docs to clarify that `new_tab()` and `switch_tab()` work in background and to use `activate_tab()` only when needed. - Syncs both packaged `browser-use` skills with `browser-harness` v0.1.9 docs. - After merge: publish the `browser-use` 0.13.8 GitHub release. No migration actions. <sup>Written for commit fa57d34. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/5481?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
Release 0.13.7 with Browser Harness 0.1.8 (#5308) Bumps the CLI's pinned harness to [browser-harness 0.1.8](https://github.com/browser-use/browser-harness/releases/tag/v0.1.8) and bumps browser-use to 0.13.7. ### What 0.1.8 brings to the CLI - Chrome no longer has to be open first — if no Chromium-family browser is running, the harness launches one, preferring a profile that already has remote debugging enabled and skipping the profile picker. - Attaches to a reusable tab (about:blank / New Tab / an existing `chrome://inspect` tab) instead of always opening a new one: faster startup, fewer prompts. - Only closes leftover `chrome://inspect` tabs the harness itself opened — user tabs are left alone. - Guards against stale DevToolsActivePort files; clearer, actionable setup/permission errors. ### Changes - `pyproject.toml`: `version` 0.13.6 → 0.13.7, `browser-harness` 0.1.6 → 0.1.8. - Both `SKILL.md` copies re-synced from browser-harness main via `scripts/sync_browser_harness_skill.py` (picks up the auto-launch note, the "When Not to Use" section, and the Allow-popup retry gotcha). `scripts/sync_browser_harness_skill.py --check` passes; `tests/ci/test_browser_use_skill_install_docs.py` and `tests/ci/test_browser_use_cli.py` pass (6/6) locally. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Release `browser-use` 0.13.7 and pin `browser-harness` 0.1.8 to improve startup and reliability: auto-launch Chrome if none is running, reuse existing tabs for faster attach, avoid closing user tabs, and handle stale DevTools ports more clearly. Synced SKILL docs with a new “When Not to Use” section, auto-launch notes, and guidance to avoid looping on the “Allow remote debugging?” popup. - **Dependencies** - `browser-use` → 0.13.7 - `browser-harness` → 0.1.8 (was 0.1.6) <sup>Written for commit 449e9ad. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/5308?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
Release 0.13.6 with Browser Harness 0.1.6 (#5245) ## Summary - bump `browser-use` from `0.13.5` to `0.13.6` - pin `browser-harness==0.1.6` - sync both Browser Use skill copies from Browser Harness `main`, adding the recordings and video workflow ## Validation - `uv run --frozen pre-commit run --files pyproject.toml skills/browser-use/SKILL.md browser_use/skills/browser-use/SKILL.md` - `uv run --frozen python scripts/sync_browser_harness_skill.py --check` - built the wheel and verified its metadata requires `browser-harness==0.1.6` - verified the packaged skill contains the recordings/video instructions <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Release `browser-use` 0.13.6 and pin `browser-harness` 0.1.6. Adds recordings and video workflow guidance to the Browser Use skill so users can opt into local traces and create videos. - **Dependencies** - Bump `browser-use` to `0.13.6`. - Pin `browser-harness` to `0.1.6`. - **New Features** - Synced both Browser Use skill docs from `browser-harness` `main`. - Added recordings/video workflow: CLI enable/disable, `BH_RECORD` override, `start_recording`/`stop_recording`, and safe use of `recordings --latest`. - Included `make-video.md` in Interaction Skills. <sup>Written for commit 169ebee. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browser-use/browser-use/pull/5245?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
PreviousNext