feat(seo): cover every docs page in /llms.txt and /llms-full.txt

[nik13]

Summary

Both src/pages/llms.txt.ts and src/pages/llms-full.txt.ts walked only tabNavigation. Any .mdx page on disk that isn't curated into navigation.ts was silently missing from the LLM-facing surfaces. A quick count showed ~83 pages on disk but not in nav — mostly cookbooks, release-notes follow-ups, and recently added pages.

Change

After this change both endpoints:

1. Still emit the navigation-driven sections first (preserves the human-grouped section structure that AI crawlers can use as semantic anchors).
2. Then sweep src/pages/docs/**/*.mdx and emit anything not yet linked:
   • In llms.txt: under an ### Additional Pages block at the end of ## Docs.
   • In llms-full.txt: appended after the nav-driven entries with the same ## Title / URL / content structure.
3. Use the page's frontmatter title: when present; fall back to title-cased slug.

Why

• New cookbook entries, release notes, FAQ updates, or any newly added doc page show up on the next deploy without needing a separate navigation.ts PR.
• The cost is zero: the filesystem walk runs at build time, same as the existing nav walk.
• Authoring stays the same — humans still curate navigation order; this is the safety net.

Test plan

• npx tsc --noEmit clean on both endpoints
• After deploy, curl https://docs.futureagi.com/llms.txt | grep -c "Additional Pages" returns 1
• Spot-check a few cookbook URLs (e.g. /docs/cookbook/gepa-optimization) appear in /llms.txt and /llms-full.txt

🤖 Generated with Claude Code