unify find-latest-version

[syphar]

Fixes #3290 by unifying "latest version" logic across all places.

This also leads to some behavioural changes, which are mostly fine IMO, they become tricky when all releases of a crate are yanked. In this "all-yanked" case, we will start treating a crate with only yanked releases as "hidden / deleted" in some cases. Personally this feels more correct, but I'm happy to discuss.

AI summary of the before/after:

Behavioural changes in fix-latest-release

This branch consolidates three "latest release" decisions onto a single canonical picker:

• /crate/{name}/latest/... URL resolution (via match_version)
• The rustdoc topbar's is_latest_version / "Go to latest" button
• The crates.latest_version_id column (read by sitemaps, home page lists, crates.io search bridge, "I'm feeling lucky", watcher rebuilds, repository stats)

The picker is:

The highest non-yanked, non-in-progress release, preferring stable over prerelease. None otherwise.

The table below shows how each surface changes per scenario.

Legend

	meaning
main	behaviour on main
this branch	behaviour after the fix
🐛	bug / inconsistency that's being fixed
✅	strict improvement
➖	intentional behaviour change, arguably an improvement
✔	no change

---

Scenario 1 — All releases yanked

Surface	main	this branch
/crate/foo/latest/...	404	404 ✔
Topbar latest_release	Returns highest yanked release → Result::? propagates "crate without releases" → 500 🐛	Returns None; "Go to latest" button hidden ✅
crates.latest_version_id	Points at a yanked release id	NULL ✅
Sitemap	Crate advertised with yanked URL → 404 on crawl 🐛	Crate dropped from sitemap ✅
Home page lists (Recent, Stars, …)	Yanked release surfaced as "latest" 🐛	Crate dropped ✅
crates.io search bridge	Linked to yanked latest	"Documentation not available on docs.rs" ➖
"I'm feeling lucky"	Could land on yanked docs	Crate skipped ✅
Watcher rebuilds	Rebuilds the yanked release	Crate skipped ✅
Repository crate_count	Counted	Not counted ➖

Scenario 2 — Yanked stable + non-yanked prerelease (the bug-report case)

Example: [0.2.0 yanked, 0.2.0-pre.1 not yanked].

Surface	main	this branch
/crate/foo/latest/...	Resolves to prerelease (0.2.0-pre.1)	Resolves to prerelease ✔
Topbar latest_release	Yanked stable (0.2.0) 🐛	Prerelease ✅
→ is_latest_version on /0.2.0-pre.1/	false → "Go to latest" button shown 🐛	true → button hidden ✅
Clicking "Go to latest"	Redirects to /latest/ → same prerelease page (loop) 🐛	Button not shown ✅
crates.latest_version_id	Yanked stable	Prerelease ✅
Home page lists	Lists the yanked stable as latest	Lists the prerelease ✅

Scenario 3 — Only non-yanked releases are in-progress

Example: brand-new crate with one release whose build hasn't finished.

Surface	main	this branch
/crate/foo/latest/...	Resolves to in-progress release → no rustdoc HTML in storage → effective 404 🐛	404 ✅
Topbar latest_release	None → 500 🐛	None → button hidden ✅
crates.latest_version_id	NULL	NULL ✔
^1.0 semver query (range-matching, not /latest/)	Returns in-progress release if it matches	Returns in-progress release if it matches ✔

Scenario 4 — Failed build is the highest non-yanked release

Surface	main	this branch
/crate/foo/latest/...	Resolves to the failed release	Resolves to the failed release ✔
Topbar latest_release	Failed release	Failed release ✔
latest_path (button target)	crate_details_url() (build wasn't successful)	crate_details_url() ✔
crates.latest_version_id	Points at failed release	Points at failed release ✔
Sitemap	Filtered out (rustdoc_status = false)	Filtered out ✔
Home page Recent / Most stars	Shown	Shown ✔
Home page "Recent failures"	Shown	Shown ✔

No change for failed-build handling — the filter on every code path has always been build_status != InProgress, which lets both Success and Failure through.

Scenario 5 — Specific yanked version (direct exact-match URL)

Example: /crate/foo/0.2.0/... where 0.2.0 is yanked.

Surface	main	this branch
Resolves the release	Yes (exact-match bypasses the picker)	Yes ✔
Yanked warning in topbar	Shown	Shown ✔
is_latest_version	Computed against the picker's choice — may or may not be the yanked version	Computed against the picker's choice ✔ (but now consistent with /latest/)

The exact-match path through match_version has never applied the yanked filter — yanked versions remain navigable when their version is in the URL. Unchanged.

---

Summary

• 🐛 → ✅ : The "topbar 500 on all-yanked" and "Go to latest leads to a same-page loop" bugs are fixed.
• 🐛 → ✅ : Sitemaps stop advertising URLs that 404.
• 🐛 → ✅ : Home page release feeds stop surfacing yanked-as-latest.
• ➖ : crates.io search bridge renders all-yanked crates as "not available" instead of linking to a yanked release. Debatable; we considered this acceptable.
• ✔ : Failed builds, direct exact-version URLs, and non-STAR semver queries (^1.0, etc.) are unchanged.

The picker is now structurally the same across /latest/, the topbar, and latest_version_id — disagreement isn't just unlikely, it's impossible by construction.

Backfill

Existing crates.latest_version_id rows on production may still point at yanked or otherwise stale picks. They self-heal next time any build event triggers update_latest_version_id for that crate. For an eager rewrite, run the existing docs_rs_admin backfill command.