Default to Markdown. It's cheaper to tokenize, cleaner to embed, and easier to diff, so it should be your baseline for ingestion, RAG, and most LLM output. Switch to HTML only when a human needs to review or interact with a long, complex artifact where layout and navigation matter. When an agent needs to click, fill, or act, skip both and reach for a structured format like SOM or JSON.
TL;DR:
- Token counts for web pages are 5 to 8 times lower with Markdown compared to raw HTML, significantly reducing ingestion costs.
- Markdown provides cleaner embeddings, improving retrieval precision by stripping attribute noise and boilerplate from page content.
- HTML is preferable for human review, long documents, and interactive dashboards due to its support for navigation, diagrams, and layout features.
- Using structured formats like SOM can be 3 to 7 times cheaper than HTML for tasks involving interaction and can parse faster for navigation.
- Defaulting to Markdown for ingestion is recommended unless the task explicitly requires interactivity or detailed layout, which favors HTML or structured JSON.
Table of Contents
- HTML vs Markdown LLM Token Costs: What the Numbers Say
- Why Markdown Wins for Ingestion and RAG Pipelines
- When Does HTML Beat Markdown for LLM Outputs?
- SOM and Structured JSON: A Third Option Worth Knowing
- Markdown vs SOM vs HTML: A Quick Decision Table
- How Do You Decide Between HTML and Markdown for an LLM Pipeline?
- How We Think About Format Choices at Gyrence
- Try Gyrence for Markdown-First Web Data
- Where to Read More on This
- Sources
- FAQ
HTML vs Markdown LLM Token Costs: What the Numbers Say
Token cost is where this argument gets settled fast. Empirical tests across 20 real web pages found Markdown uses roughly 5 to 8 times fewer tokens than raw HTML on average, with news pages sometimes showing reductions above 10x. Already-clean HTML, like MediaWiki output, shows smaller gains because there's less markup noise to strip out in the first place.
Where does the overhead come from? A few repeat offenders show up in nearly every measurement:
- Inline
<script>and<style>blocks that never made it to a CDN - JSON-LD schema blobs duplicating content already visible on the page
- Utility class soup (
class="flex items-center gap-2 text-sm...") repeated on every element - Inline SVG icons and decorative markup with no semantic payload
Statistic Callout: Markdown conversion cuts token counts by 5 to 8x on average across tested page types, which is the single biggest lever most teams have for cutting LLM ingestion costs without touching model choice.
Run that ratio against volume and the dollar signs get real. A RAG pipeline chunking tens of thousands of documents a month can see annual savings ranging from the low thousands to tens of thousands of dollars just from converting before embedding, depending on traffic and chunk size. The caveat: these ratios shift by page type, and a small internal wiki page might not justify the conversion overhead at all.
Why Markdown Wins for Ingestion and RAG Pipelines
Markdown's real advantage isn't just fewer tokens. It's cleaner embeddings. Stripped of attribute noise and boilerplate, a Markdown chunk embeds closer to its actual semantic content, which improves retrieval precision in vector search. HTML's tag soup dilutes the signal your embedding model is trying to capture.
It also fits how developers already work:
- Git diffs stay readable instead of turning into unreadable tag-attribute noise
- CI pipelines and text tooling (grep, sed, linters) work without an HTML parser in the loop
- Headings, code blocks, and GFM tables preserve document structure without needing a DOM
Markdown does lose things HTML keeps: nested tabs, collapsible sections, inline widgets. For most ingestion jobs, you don't need them.
Pro Tip: When converting HTML tables to Markdown, watch for merged cells and nested tables. Most converters flatten them silently, which quietly corrupts the data your agent later retrieves.
Our own developer guide to fetching web pages as markdown walks through the conversion steps and the token-estimation math in more detail.
When Does HTML Beat Markdown for LLM Outputs?
Markdown's flatness becomes a liability once a human is the one reading the output. Long specs, multi-section reports, and interactive dashboards need affordances Markdown simply doesn't have.
HTML gives you:
- SVG diagrams and charts rendered inline, not described in prose
- Tabs and collapsible sections that let reviewers skip to the relevant part
- In-page navigation (anchor links, a table of contents) for documents running past a few thousand words
- Fine-grained layout control for side-by-side comparisons or annotated code
This lines up with what Anthropic's engineering lead has argued: HTML is increasingly better than Markdown at keeping humans engaged in agentic loops, specifically because long agent outputs need structure a human can scan instead of scroll through linearly. A parallel argument from Simon Willison's writeup on Claude Code makes the same case: HTML artifacts keep reviewers in the loop longer because the format supports diagrams and navigation Markdown can't.
Accepting HTML output does carry risk. Sanitize aggressively, strip script tags, and never render agent-generated HTML in a privileged context without a content security policy.
SOM and Structured JSON: A Third Option Worth Knowing
Semantic Object Model output tags each element with a role and explicit action (click, fill, navigate) plus metadata like coordinates or ARIA labels, instead of rendering full markup or flattening everything to prose. It's built for agents that act, not agents that summarize.
The numbers back it up. A three-way comparison of HTML, Markdown, and SOM found SOM can run 3 to 7 times cheaper than raw HTML while parsing faster than Markdown on navigation tasks, since the agent isn't guessing which flattened text block corresponds to a clickable button.
Build SOM output when:
- Your agent needs to interact with a page, not just read it
- Markdown's flattening destroys the actions available on the source page
- Latency matters more than expressiveness (SOM parses faster than ambiguous Markdown link lists)
The tradeoff is tooling maturity. SOM libraries are younger than either HTML or Markdown parsers, so expect to build more of the extraction logic yourself.
Markdown vs SOM vs HTML: A Quick Decision Table
| Task | Best format | Why |
|---|---|---|
| Summarize an article | Markdown | Lowest tokens, clean embeddings |
| Interactive workflow (click/fill) | SOM | Explicit actions, low latency |
| Long spec or human review | HTML | Navigation, diagrams, layout |
| Site navigation / crawling | SOM | Faster parsing than ambiguous links |
Three questions settle most format decisions:
- What's the job? Reading, acting, or presenting to a human?
- How cost-sensitive is this pipeline at your expected volume?
- Does the task require interactivity Markdown can't express?
One tuning note: estimate tokens before you commit to a chunking strategy, and check whether your target site or API supports Accept: text/markdown negotiation before building a custom scraper.
How Do You Decide Between HTML and Markdown for an LLM Pipeline?
Four rules cover almost every real pipeline decision:
- Text ingestion or RAG? Default to Markdown. It's cheaper and embeds cleaner.
- Human-facing specs or interactive reports? Use HTML. The reviewability gain outweighs the token cost.
- Agent needs to click, fill, or navigate? Use SOM or structured JSON. Nothing else gives you explicit actions.
- Not sure yet? Convert to Markdown by default, but preserve structured metadata (links, table structure, headings) alongside it so you're not locked in.
Pro Tip: Store both raw HTML and a Markdown summary during your first few weeks in production. You'll have the data to prove which format actually reduced errors instead of guessing after the fact.
Cloudflare's Markdown for Agents feature is worth studying here. It serves Markdown via content negotiation and includes token-estimate headers, which is exactly the kind of production pattern worth copying into your own middleware.
How We Think About Format Choices at Gyrence
We default to Markdown-first ingestion because that's what most pipelines need most of the time. When a task calls for interactivity, we offer structured extraction instead of forcing raw HTML into a prompt. Conversion controls and spending caps matter because converting thousands of pages without a cost ceiling is how teams get surprised by a bill. Run a small batch through both formats on your own pages before deciding. The token counts alone usually make the call for you.
— Glen
Try Gyrence for Markdown-First Web Data
If this comparison convinced you Markdown should be your default, the next question is how you get there without hand-rolling a converter. Gyrence's Fetch primitive pulls a page and cleans it to Markdown directly, so you're not writing your own HTML stripper. When a task needs structure instead of just prose, Extract turns the same page into schema-guided JSON using LLM-powered extraction, no separate line item on your bill. Both are reachable through the API or the hosted MCP endpoint, with spending caps that keep a large conversion job from turning into a surprise invoice.
Check the pricing page to compare the Standard, Growth, and Scale tiers against your expected page volume, or read the developer docs to see the Fetch and Extract endpoints in practice. If you're also watching pages for changes over time, WebDoppler adds monitoring and webhook alerts on top of the same pipeline. Start with the free tier and run your own token count on a handful of pages before committing to a plan.
Where to Read More on This
For deeper technical detail beyond what's covered here:
- Simon Willison's case for HTML in agent-to-human workflows
- The DEV Community breakdown of HTML vs Markdown vs SOM with latency figures
- Cloudflare's Markdown for Agents technical note on content negotiation
- Gyrence's own guide to structuring scraped data for agent tasks
If you're scaling ingestion further, proxy strategy for large-scale AI training data collection is worth a look before you hit rate limits.
Sources
- HTML vs Markdown: Token Count on 20 Real Web Pages
- Using Claude Code: The Unreasonable Effectiveness of HTML
- HTML vs Markdown vs SOM: Which Format Should Your AI Agent Use? - DEV Community
- Introducing Markdown for Agents - Cloudflare Blog
FAQ
Is Markdown Better Than HTML for LLMs?
For ingestion and RAG, yes. Markdown uses 5 to 8 times fewer tokens on average and produces cleaner embeddings. HTML wins when a human needs to review a long, complex output with navigation or diagrams.
Why Is Markdown Better for LLMs Specifically?
Markdown strips out the tag attributes, scripts, and styling noise that make up most of an HTML page's token count. That leaves more of your context window for actual content, and it embeds closer to the true semantic meaning of the text.
Is HTML Still Used for LLM Outputs?
Yes, and its use case is narrowing rather than shrinking. HTML remains the better choice for agent-to-human artifacts like long specs, interactive dashboards, and diagram-heavy reports, per the Anthropic engineering argument that HTML keeps reviewers engaged in long agentic loops.
Is HTML Better Than Markdown for AI Development in General?
Not universally. HTML is better when human review and interactivity matter more than cost. Markdown is better when token efficiency and clean ingestion matter more, which covers most RAG and agent-input pipelines.
What Does Gyrence Charge for Fetch-to-Markdown Conversion?
Gyrence's plans start with a Free tier and a Pay-As-You-Go option, with Standard priced at $75 per month, Growth at $299 per month, and Scale at $549 per month. Full details are on the pricing page.