Blog
Discover our latest articles and blogs
Boilerplate Removal: How It Works and Where It Fails
Boilerplate removal strips navigation, ads and footers so you keep the main content. Learn the methods, their failure modes and how to test them for RAG.
Unicode Normalization for Scraped Web Data: NFC vs NFKC
How to apply Unicode normalization to scraped web data: which form to pick (NFC, NFKC), what it breaks, and where it fits before dedup and language detection.
Claude MCP Scraping: Set Up Web Tools the Safe Way
Connect Claude Code and Claude Desktop to a scraping MCP server: transports, config pitfalls, token costs, spending caps and failure handling, step by step.
What Makes a Scraper LLM-Friendly (and How to Test One)
An LLM-friendly scraper returns clean markdown, typed JSON and honest failures. Here are six criteria to test any scraper against, plus the legal basics.
MCP Web Scraping: How to Give Agents the Web Safely
MCP web scraping for AI agents: how it works, the security risks of raw HTML, how to cap costs, and when a plain HTTP fetch beats an agent loop.
Hosted MCP Endpoint: What It Is and When to Use One
A hosted MCP endpoint is a remote MCP server your agent calls over HTTPS. Learn how it works, what to secure, and when it beats a local server.
Search Operators for Research: A Working Guide
Boolean logic, phrase search, truncation and Google operators for research, plus how syntax differs by database and how to log strings so they can be rerun.
LlamaIndex MCP: Connect Agents to Web Data Tools
How to use MCP with LlamaIndex: install llama-index-tools-mcp, pick a transport, load tools into an agent, and handle web data failures without guessing.
MCP vs Function Calling: How They Fit Together
MCP and function calling work at different layers. See what each does, when to use which, and how to combine them in a production agent.