SSH into any website

What is OpenWeb?

OpenWeb crawls websites and converts them to plain markdown files you can search with standard Unix commands. No SDK, no API client — just bash.

• [1]
  Explicit. Every crawled page is a readable markdown file on a navigable filesystem. You can see exactly what's been indexed, inspect it, search it, and manage it — nothing is hidden in a black-box embedding store.
• [2]
  File over app. Web content is stored as plain markdown files. That means it's interoperable — pipe it through grep, jq, awk, or any Unix tool. Feed it to any AI. The entire Unix toolkit just works.
• [3]
  No SDK, simple bash. No client library, no API wrapper, no dependency. Just ssh openweb.sh — cat, grep, find, diff. If your agent can write bash, it can use OpenWeb.
• [4]
  BYOAI. Plug any AI into this data — Claude, GPT, Gemini, open-source models, your own fine-tune. OpenWeb gives you structured web content and a personal filesystem; you choose what reasons over it.
• [5]
  Yours. Your crawled data lives in your session, accessible via your SSH key. It's not locked inside a proprietary system — you can extract, copy, and move it however you want. You control your information.
• [6]
  Collective index. No need to burn compute crawling or spend tokens converting HTML to markdown. The web is already mounted as a shared drive — if someone indexed a page before you, it's instantly available.

How it works

Three steps from zero to searching any website's content.

Use cases

• [*]
  Knowledge base Build a personal or company knowledge base from any set of websites — docs, wikis, blogs — all as searchable markdown files your AI agents can reason over
• [*]
  Personalized onboarding Crawl a user's website to tailor their first-run experience with relevant context
• [*]
  AI CMO Let marketing agents research competitor sites, track messaging changes, and draft positioning
• [*]
  Competitive intelligence Agents compare feature pages across competitors to surface gaps and opportunities
• [*]
  Scheduled signal tracking Monitor websites on a cron for pricing changes, new features, or content updates