macOSWindowsiOSAndroidHub
Site analyze and selective ingest
Discover pages across a whole site, preview titles and quality hints, then ingest only what you need — no live mirror of the site.
Before you start
- Signed in with a Mind notebook open.
- Root URL must be https (or http) with same-origin links only.
Steps
- 1
Mind → notebook → Sources (Hub) or Add sources → Websites (desktop/mobile).
- 2
Enter the site root URL (e.g. https://docs.example.com) → Analyze site.
- 3
Server reads robots.txt, sitemap.xml when present, then BFS same-origin crawl up to plan limit.
- 4
Preview lists title, page type, quality hint — toggle pages on or off.
- 5
Ingest selected sends only chosen URLs; each becomes a normal web source (skip_analyze on server).
- 6
Single Add website still ingests one URL immediately without the analyze step.
- 7
Failed URLs show in source status — Reprocess from the source row.
Common mistakes
- Trial discovers up to ~25 pages and ingests ~10 per batch; Pro allows larger discovery and ingest.
- Prefer analyze for docs sites; use site-pack paste for a hand-picked URL list.
- Chat citations use indexed text — we do not claim a live mirror of the site.