Rover

Why Rover

Agents hit the same four walls on the live web.

Boilerplate buries the content

Ads, nav, and cookie banners drown the real text, and your token budget pays for every byte of it.

JavaScript hides the page

A modern SPA hands a non-browser an empty <div id="root">, so the model reasons over a shell.

Repeated fetches cost real money

Hammering the same URL burns tokens, time, and money, and ignores the politeness rules every host expects.

Fetched content is hostile

A page can carry “ignore your instructions” straight into the context window, and most fetch tools hand it over raw.

The output

A document, not a one-shot answer.

Rover returns a Markdown document with YAML frontmatter: the content hash, the token count, the extraction-quality score. Cache it, re-read it, diff it against the next fetch. It stays the same document instead of a fresh model answer every prompt.

---
url: "https://en.wikipedia.org/wiki/Rust_(programming_language)"
title: "Rust (programming language) - Wikipedia"
content_hash: "sha256:b3e9…"
estimated_tokens: 14823
tokenizer: "o200k"
extraction_quality: 0.98
---

Frontmatter travels with the content.

How it compares

Rover vs WebFetch vs wget

	Rover	WebFetch	wget
What you get back	✅ Clean Markdown document	◻️ Lossy per-prompt answer	❌ Raw HTML
Reusable across calls	✅ Cached, stable hash	❌ Re-runs the model	✅ Raw file
Token budgeting & counts	✅	❌	❌
HTTP-aware caching	✅ TTL / ETag / SWR	◻️ Flat 15-min	◻️ Timestamping
SSRF / private-network guard	✅ 5 levels + dial-time recheck	◻️	❌
Prompt-injection guard	✅ Layered	❌	—

✅ full · ◻️ partial · ❌ none · — not applicable

The tools

Five tools your agent gets on day one.

fetch

Turn a single URL into cleaned Markdown, with caching, headless rendering, image modes, token budgeting, and inline summarisation.

batch_fetch

Fetch N URLs concurrently with per-domain rate limiting, streaming NDJSON progress as each one lands.

summarize

Compact a page through an extractive offline backend or a cloud one, steered by focus, preserve, and target_tokens.

get_metadata

Pull Schema.org, Open Graph, and Twitter Card metadata without fetching the full body.

count_tokens

Estimate a URL’s token cost across cl100k, o200k, claude, llama3, and qwen3 before you pay for it.

Security & trust

The web is hostile by default. Rover treats it that way.

Prompt-injection guard

A per-response nonce wrapper holds by construction, backed by a pattern detector and an optional ONNX classifier for the phrasings rules miss.

SSRF protection

Five policy levels plus a dial-time re-check that re-validates every resolved address before the socket opens, closing the DNS-rebinding window.

Secret redaction

URL secrets and Authorization credentials are scrubbed before any event reaches a log.

Short cache TTL

The 15-minute default keeps the blast radius small when content is poisoned or changes underneath you.

Wire it into your agent.

claude mcp add rover -- rover mcpRead the docs →