From 97e27b76e2d59525c823e16933763fca89e18978 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 06:40:18 -0700 Subject: [PATCH 1/7] =?UTF-8?q?=E2=9C=A8=20feat(discovery):=20add=20iter?= =?UTF-8?q?=5Finterpreters=20for=20enumeration?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Callers wanting every interpreter on the system, rather than the first match, had to abuse get_interpreter's predicate as a side-channel: return False for every candidate so the search never stops, accumulate into a dict keyed by realpath. The workaround missed any interpreter whose binary on PATH is named pypy or graalpy, and surfaced /bin/python and /usr/bin/python as separate entries even though they resolve to the same file. iter_interpreters yields every match in discovery order, deduplicated by the resolved real path of system_executable so symlinked aliases and venvs that symlink to a base interpreter collapse to a single entry. With no spec it broadens the PATH and uv-install regex to every name in KNOWN_IMPLEMENTATIONS, because an "all interpreters" call that quietly drops PyPy and GraalPy would just shift the gotcha. get_interpreter's narrow regex stays as it was so tools that have always read "no implementation in the spec" as "give me CPython" keep that behaviour. Closes #65. --- docs/changelog/65.feature.rst | 3 + docs/explanation.rst | 39 ++++++++ docs/how-to/standalone-usage.rst | 54 ++++++++++ docs/tutorial/getting-started.rst | 43 ++++++++ src/python_discovery/__init__.py | 6 +- src/python_discovery/_discovery.py | 100 ++++++++++++++++--- src/python_discovery/_py_spec.py | 14 ++- tests/test_iter_interpreters.py | 152 +++++++++++++++++++++++++++++ 8 files changed, 396 insertions(+), 15 deletions(-) create mode 100644 docs/changelog/65.feature.rst create mode 100644 tests/test_iter_interpreters.py diff --git a/docs/changelog/65.feature.rst b/docs/changelog/65.feature.rst new file mode 100644 index 0000000..412641f --- /dev/null +++ b/docs/changelog/65.feature.rst @@ -0,0 +1,3 @@ +add :func:`~python_discovery.iter_interpreters` for enumerating every discovered interpreter, with PATH and +UV-install support for non-CPython implementations listed in :data:`~python_discovery.KNOWN_IMPLEMENTATIONS` +- by :user:`gaborbernat`. diff --git a/docs/explanation.rst b/docs/explanation.rst index 2d1bc22..cfbf8f4 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -65,6 +65,45 @@ detects these shims and resolves them to the actual binary. `mise `_ and `asdf `_ work similarly, using the ``MISE_DATA_DIR`` and ``ASDF_DATA_DIR`` environment variables to locate their installations. +Selecting one interpreter vs. enumerating all of them +------------------------------------------------------- + +:func:`~python_discovery.get_interpreter` and :func:`~python_discovery.iter_interpreters` walk the same candidate +sources, but they answer different questions and behave differently in three ways. + +.. mermaid:: + + flowchart LR + Sources["candidate sources
(try_first_with → current →
PEP 514 → PATH → uv)"] + Sources --> Get["get_interpreter()
first match wins, returns one"] + Sources --> Iter["iter_interpreters()
yields every match"] + + style Get fill:#4a9f4a,stroke:#2a6f2a,color:#fff + style Iter fill:#4a90d9,stroke:#2a5f8f,color:#fff + +**Implementation coverage on PATH.** :func:`~python_discovery.get_interpreter` matches only ``python*`` filenames on +PATH unless the spec names another implementation explicitly (``pypy3.12``, ``graalpy3.11``). This keeps backwards +compatibility with tools that have always read "no implementation in the spec" as "give me CPython." +:func:`~python_discovery.iter_interpreters` with no spec broadens the search to every name in +:data:`~python_discovery.KNOWN_IMPLEMENTATIONS` -- otherwise an "all interpreters" call would silently miss every +PyPy and GraalPy on the system. When you pass a spec to :func:`~python_discovery.iter_interpreters`, it falls back +to the same narrow regex as :func:`~python_discovery.get_interpreter`, so behaviour is consistent across the two +APIs whenever a spec is given. + +**Deduplication.** :func:`~python_discovery.get_interpreter` deduplicates per call so it does not interrogate the +same binary twice while searching, and stops as soon as a match is found. :func:`~python_discovery.iter_interpreters` +deduplicates by the resolved real path of each candidate's ``system_executable`` (falling back to ``executable``). +That means symlinked aliases like ``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv whose ``python`` +symlinks to its base interpreter, collapse to a single yield. The semantic is "one entry per distinct install," +which is what callers building choosers or version-range pickers usually want. + +**Iteration order.** Yields come back in *priority order*: ``try_first_with`` first, then the running interpreter, +then PEP 514 entries on Windows, then PATH left-to-right, then UV-managed installs. This matches what +:func:`~python_discovery.get_interpreter` would have returned at each step. If your ordering differs (newest +version first, smallest install root, etc.), wrap the call in :func:`sorted` -- the API deliberately does not +include a ``sort_by`` parameter because keeping discovery order preserves the priority signal for callers who +want it. + How caching works ------------------- diff --git a/docs/how-to/standalone-usage.rst b/docs/how-to/standalone-usage.rst index adf151e..507cdd2 100644 --- a/docs/how-to/standalone-usage.rst +++ b/docs/how-to/standalone-usage.rst @@ -62,6 +62,60 @@ the ``PY_DISCOVERY_TIMEOUT`` environment variable. The timeout value should be a number in seconds. Each interpreter candidate is given this much time to respond. If a timeout occurs, the candidate is skipped and the search continues with the next one. +List every interpreter on the system +-------------------------------------- + +Use :func:`~python_discovery.iter_interpreters` to enumerate every Python python-discovery can find. With no spec +it yields all known implementations (CPython, PyPy, GraalPy -- see +:data:`~python_discovery.KNOWN_IMPLEMENTATIONS`). Pass a spec to filter, exactly like +:func:`~python_discovery.get_interpreter`. + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + + # Every interpreter, no filter + for info in iter_interpreters(cache=cache): + print(info.executable, info.version_str, info.implementation) + + # Every CPython 3.10 or newer, newest first + newest_first = sorted( + iter_interpreters("cpython>=3.10", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + +Enumeration interrogates every candidate as a subprocess on a cold cache. Always pass a +:class:`~python_discovery.DiskCache` if you call this more than once. + +Pick an interpreter from a range, preferring newer +---------------------------------------------------- + +A common need: search a version range and prefer newer interpreters when more than one matches. Sort the result of +:func:`~python_discovery.iter_interpreters` and take the first. + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + + matches = sorted( + iter_interpreters("cpython>=3.10,<3.15", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + info = matches[0] if matches else None + +Use :func:`get_interpreter` instead when you only need the first PATH-priority hit; use the sort-and-take pattern +when *your* ordering differs from PATH order (newest version, smallest install size, preferred install root, etc.). + Read interpreter metadata --------------------------- diff --git a/docs/tutorial/getting-started.rst b/docs/tutorial/getting-started.rst index 696efcf..5f1c3ca 100644 --- a/docs/tutorial/getting-started.rst +++ b/docs/tutorial/getting-started.rst @@ -87,6 +87,49 @@ You can pass multiple specs as a list -- the library tries each one in order and result = get_interpreter(["python3.12", "python3.11"], cache=cache) +Listing every interpreter +--------------------------- + +When you need *every* interpreter rather than just the first match -- for example, to show the user a chooser, or +to apply your own ranking -- use :func:`~python_discovery.iter_interpreters`. Pass no arguments to enumerate every +implementation python-discovery knows about, or pass a spec to filter. + +.. mermaid:: + + flowchart TD + Call["iter_interpreters(spec, cache)"] --> Yield["yields PythonInfo"] + Yield --> A["1. try_first_with paths"] + Yield --> B["2. running interpreter"] + Yield --> C["3. PATH (left to right)"] + Yield --> D["4. uv-managed installs"] + + style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff + style Yield fill:#4a9f4a,stroke:#2a6f2a,color:#fff + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + for info in iter_interpreters(cache=cache): + print(info.executable, info.version_str, info.implementation) + +The result is an iterator, so :func:`list`, :func:`sorted`, generator expressions and early ``break`` all work as you +would expect. Symlinked aliases (``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv and the base it points at) +collapse to a single entry, so you do not see the same install twice. + +To prefer newer interpreters in a range, sort the result by ``version_info`` after filtering: + +.. code-block:: python + + newest_first = sorted( + iter_interpreters(">=3.10,<3.15", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + Writing specs ------------- diff --git a/src/python_discovery/__init__.py b/src/python_discovery/__init__.py index 40307ae..4533386 100644 --- a/src/python_discovery/__init__.py +++ b/src/python_discovery/__init__.py @@ -5,15 +5,16 @@ from importlib.metadata import version from ._cache import ContentStore, DiskCache, PyInfoCache -from ._discovery import get_interpreter +from ._discovery import get_interpreter, iter_interpreters from ._py_info import KNOWN_ARCHITECTURES, PythonInfo, normalize_isa -from ._py_spec import PythonSpec +from ._py_spec import KNOWN_IMPLEMENTATIONS, PythonSpec from ._specifier import SimpleSpecifier, SimpleSpecifierSet, SimpleVersion __version__ = version("python-discovery") __all__ = [ "KNOWN_ARCHITECTURES", + "KNOWN_IMPLEMENTATIONS", "ContentStore", "DiskCache", "PyInfoCache", @@ -24,5 +25,6 @@ "SimpleVersion", "__version__", "get_interpreter", + "iter_interpreters", "normalize_isa", ] diff --git a/src/python_discovery/_discovery.py b/src/python_discovery/_discovery.py index d534598..f676d2e 100644 --- a/src/python_discovery/_discovery.py +++ b/src/python_discovery/_discovery.py @@ -11,10 +11,10 @@ from ._compat import fs_path_id from ._py_info import PythonInfo -from ._py_spec import PythonSpec +from ._py_spec import KNOWN_IMPLEMENTATIONS, PythonSpec if TYPE_CHECKING: - from collections.abc import Callable, Generator, Iterable, Mapping, Sequence + from collections.abc import Callable, Generator, Iterable, Iterator, Mapping, Sequence from ._cache import PyInfoCache @@ -51,6 +51,64 @@ def get_interpreter( return None +def iter_interpreters( + key: str | Sequence[str] | None = None, + try_first_with: Iterable[str] | None = None, + cache: PyInfoCache | None = None, + env: Mapping[str, str] | None = None, + predicate: Callable[[PythonInfo], bool] | None = None, +) -> Iterator[PythonInfo]: + """ + Yield every interpreter on the system that satisfies *key*. + + Iteration order is discovery order: ``try_first_with`` paths first, then the running interpreter, then ``PATH`` + (left to right), then UV-managed installs. Results are deduplicated by the resolved real path of the underlying + system interpreter, so symlinked aliases (``/bin`` vs ``/usr/bin``) and venvs that symlink to a base interpreter + collapse to a single entry. Callers that want a different ordering should sort the result. + + :param key: interpreter specification — same syntax as :func:`get_interpreter`. ``None`` enumerates every Python + implementation python-discovery knows about (see :data:`KNOWN_IMPLEMENTATIONS`). + :param try_first_with: executables to probe before the normal discovery search. + :param cache: interpreter metadata cache; when ``None`` results are not cached. Strongly recommended for + enumeration, which interrogates every candidate as a subprocess on a cold cache. + :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. + :param predicate: optional filter applied after the spec match; return ``True`` to include the interpreter. + """ + env_map = os.environ if env is None else env + try_first_tuple = tuple(try_first_with or ()) + if key is None: + keys: tuple[str | None, ...] = (None,) + elif isinstance(key, str): + keys = (key,) + else: + keys = tuple(key) + seen_real_paths: set[str] = set() + for spec_str in keys: + if spec_str is None: + spec = PythonSpec("", None, None, None, None, None, None) + wide = True + else: + spec = PythonSpec.from_string_spec(spec_str) + wide = False + for interpreter, impl_must_match in propose_interpreters( + spec, try_first_tuple, cache, env_map, all_implementations=wide + ): + if interpreter is None: + continue + anchor = interpreter.system_executable or interpreter.executable + if anchor is None: + continue + real_path = os.path.realpath(anchor) + if real_path in seen_real_paths: + continue + if not interpreter.satisfies(spec, impl_must_match=impl_must_match): + continue + if predicate is not None and not predicate(interpreter): + continue + seen_real_paths.add(real_path) + yield interpreter + + def _find_interpreter( key: str, try_first_with: Iterable[str], @@ -106,6 +164,8 @@ def propose_interpreters( try_first_with: Iterable[str], cache: PyInfoCache | None = None, env: Mapping[str, str] | None = None, + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: """ Yield ``(interpreter, impl_must_match)`` candidates for *spec*. @@ -114,6 +174,8 @@ def propose_interpreters( :param try_first_with: executable paths to probe before the standard search. :param cache: interpreter metadata cache; when ``None`` results are not cached. :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. + :param all_implementations: when ``True`` and *spec* does not constrain the implementation, also surface + non-CPython binaries on ``PATH`` and under UV's install directory. Used by enumeration APIs. """ env = os.environ if env is None else env tested_exes: set[str] = set() @@ -125,8 +187,8 @@ def propose_interpreters( yield from _propose_explicit(spec, try_first_with, cache, env, tested_exes) if spec.path is not None and spec.is_abs: # pragma: no cover # relative spec.path is never abs return - yield from _propose_from_path(spec, cache, env, tested_exes) - yield from _propose_from_uv(cache, env) + yield from _propose_from_path(spec, cache, env, tested_exes, all_implementations=all_implementations) + yield from _propose_from_uv(cache, env, all_implementations=all_implementations) def _propose_explicit( @@ -170,8 +232,10 @@ def _propose_from_path( cache: PyInfoCache | None, env: Mapping[str, str], tested_exes: set[str], + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: - find_candidates = path_exe_finder(spec) + find_candidates = path_exe_finder(spec, all_implementations=all_implementations) for pos, path in enumerate(get_paths(env)): _LOGGER.debug(LazyPathDump(pos, path, env)) for exe, impl_must_match in find_candidates(path): @@ -189,6 +253,8 @@ def _propose_from_path( def _propose_from_uv( cache: PyInfoCache | None, env: Mapping[str, str], + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: if uv_python_dir := os.getenv("UV_PYTHON_INSTALL_DIR"): uv_python_path = Path(uv_python_dir).expanduser() @@ -197,10 +263,19 @@ def _propose_from_uv( else: uv_python_path = user_data_path("uv") / "python" - for exe_path in uv_python_path.glob("*/bin/python"): # pragma: no branch - interpreter = PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env) - if interpreter is not None: # pragma: no branch - yield interpreter, True + patterns: list[str] = ["*/bin/python"] + if all_implementations: + patterns.extend(f"*/bin/{impl}" for impl in KNOWN_IMPLEMENTATIONS if impl != "python") + seen_uv_paths: set[str] = set() + for pattern in patterns: + for exe_path in uv_python_path.glob(pattern): + resolved = str(Path(exe_path).resolve()) + if resolved in seen_uv_paths: + continue + seen_uv_paths.add(resolved) + interpreter = PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env) + if interpreter is not None: # pragma: no branch + yield interpreter, True def get_paths(env: Mapping[str, str]) -> Generator[Path, None, None]: @@ -244,9 +319,11 @@ def __repr__(self) -> str: return content -def path_exe_finder(spec: PythonSpec) -> Callable[[Path], Generator[tuple[Path, bool], None, None]]: +def path_exe_finder( + spec: PythonSpec, *, all_implementations: bool = False +) -> Callable[[Path], Generator[tuple[Path, bool], None, None]]: """Given a spec, return a function that can be called on a path to find all matching files in it.""" - pat = spec.generate_re(windows=sys.platform == "win32") + pat = spec.generate_re(windows=sys.platform == "win32", all_implementations=all_implementations) direct = spec.str_spec if sys.platform == "win32": # pragma: win32 cover direct = f"{direct}.exe" @@ -331,5 +408,6 @@ class PathPythonInfo(PythonInfo): "PathPythonInfo", "get_interpreter", "get_paths", + "iter_interpreters", "propose_interpreters", ] diff --git a/src/python_discovery/_py_spec.py b/src/python_discovery/_py_spec.py index ebdea59..8c89442 100644 --- a/src/python_discovery/_py_spec.py +++ b/src/python_discovery/_py_spec.py @@ -35,6 +35,8 @@ _MAX_VERSION_PARTS: Final[int] = 3 _SINGLE_DIGIT_MAX: Final[int] = 9 +KNOWN_IMPLEMENTATIONS: Final[tuple[str, ...]] = ("python", "cpython", "pypy", "graalpy") + SpecifierSet = SimpleSpecifierSet Version = SimpleVersion InvalidSpecifier = ValueError @@ -155,16 +157,23 @@ def from_string_spec(cls, string_spec: str) -> PythonSpec: return result return cls(string_spec, None, None, None, None, None, string_spec) - def generate_re(self, *, windows: bool) -> re.Pattern: + def generate_re(self, *, windows: bool, all_implementations: bool = False) -> re.Pattern: """ Generate a regular expression for matching interpreter filenames. :param windows: if ``True``, require a ``.exe`` suffix. + :param all_implementations: when ``True`` and the spec does not constrain the implementation, match every + filename in :data:`KNOWN_IMPLEMENTATIONS` instead of only ``python``. Used by enumeration APIs. """ version = r"{}(\.{}(\.{})?)?".format( *(r"\d+" if v is None else v for v in (self.major, self.minor, self.micro)), ) - impl = "python" if self.implementation is None else f"python|{re.escape(self.implementation)}" + if self.implementation is not None: + impl = f"python|{re.escape(self.implementation)}" + elif all_implementations: + impl = "|".join(re.escape(i) for i in KNOWN_IMPLEMENTATIONS) + else: + impl = "python" mod = "t?" if self.free_threaded else "" suffix = r"\.exe" if windows else "" version_conditional = "?" if windows or self.major is None else "" @@ -254,6 +263,7 @@ def __repr__(self) -> str: __all__ = [ + "KNOWN_IMPLEMENTATIONS", "InvalidSpecifier", "InvalidVersion", "PythonSpec", diff --git a/tests/test_iter_interpreters.py b/tests/test_iter_interpreters.py new file mode 100644 index 0000000..025ac45 --- /dev/null +++ b/tests/test_iter_interpreters.py @@ -0,0 +1,152 @@ +from __future__ import annotations + +import os +import sys +from pathlib import Path +from typing import TYPE_CHECKING +from uuid import uuid4 + +from python_discovery import DiskCache, PythonInfo, iter_interpreters +from python_discovery._discovery import IS_WIN + +if TYPE_CHECKING: + from collections.abc import Iterator + + import pytest + from pytest_mock import MockerFixture + + +def test_iter_interpreters_with_abs_path_yields_one(session_cache: DiskCache) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache)) + assert len(results) == 1 + assert results[0].executable == sys.executable + + +def test_iter_interpreters_predicate_filters_all(session_cache: DiskCache) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=lambda _: False)) + assert results == [] + + +def test_iter_interpreters_predicate_keeps_all(session_cache: DiskCache) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=lambda _: True)) + assert len(results) == 1 + + +def test_iter_interpreters_no_match(session_cache: DiskCache) -> None: + results = list(iter_interpreters(uuid4().hex, cache=session_cache)) + assert results == [] + + +def test_iter_interpreters_satisfies_filter_drops_all(session_cache: DiskCache) -> None: + results = list(iter_interpreters(">=999", cache=session_cache)) + assert results == [] + + +def test_iter_interpreters_no_key_includes_running_interpreter( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + session_cache: DiskCache, +) -> None: + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + real_self = os.path.realpath(sys.executable) + results = list(iter_interpreters(cache=session_cache)) + assert any(os.path.realpath(info.executable) == real_self for info in results) + + +def test_iter_interpreters_dedups_symlinks( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + session_cache: DiskCache, +) -> None: + suffix = ".exe" if IS_WIN else "" + for name in ("python3", "python3.99"): + Path(str(tmp_path / f"{name}{suffix}")).symlink_to(sys.executable) + pyvenv_cfg = Path(sys.executable).parents[1] / "pyvenv.cfg" + if pyvenv_cfg.exists(): # pragma: no branch + (tmp_path / pyvenv_cfg.name).write_bytes(pyvenv_cfg.read_bytes()) + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + monkeypatch.setenv("PATH", os.pathsep.join([str(tmp_path), os.environ.get("PATH", "")])) + + real = os.path.realpath(sys.executable) + matches = [info for info in iter_interpreters(cache=session_cache) if os.path.realpath(info.executable) == real] + assert len(matches) == 1 + + +def test_iter_interpreters_sequence_dedups_across_keys(session_cache: DiskCache) -> None: + results = list(iter_interpreters([sys.executable, sys.executable], cache=session_cache)) + assert len(results) == 1 + + +def test_iter_interpreters_try_first_with_yields_first( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + session_cache: DiskCache, +) -> None: + monkeypatch.setenv("PATH", str(tmp_path)) + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + results = list(iter_interpreters(try_first_with=[sys.executable], cache=session_cache)) + assert results + assert os.path.realpath(results[0].executable) == os.path.realpath(sys.executable) + + +def test_iter_interpreters_returns_iterator(session_cache: DiskCache) -> None: + result: Iterator[PythonInfo] = iter_interpreters(sys.executable, cache=session_cache) + assert iter(result) is result + + +def test_iter_interpreters_skips_none_from_propose( + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + real_info = PythonInfo.current(session_cache) + mocker.patch( + "python_discovery._discovery.propose_interpreters", + return_value=iter([(None, True), (real_info, True)]), + ) + results = list(iter_interpreters(cache=session_cache)) + assert len(results) == 1 + assert results[0] is real_info + + +def test_iter_interpreters_skips_when_no_executable( + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + bogus = mocker.MagicMock(spec=PythonInfo) + bogus.system_executable = None + bogus.executable = None + mocker.patch( + "python_discovery._discovery.propose_interpreters", + return_value=iter([(bogus, True)]), + ) + results = list(iter_interpreters(cache=session_cache)) + assert results == [] + + +def test_iter_interpreters_uv_wide_pattern_finds_pypy( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + monkeypatch.setenv("PATH", "") + uv_dir = tmp_path / "uv-python" + uv_dir.mkdir() + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) + + pypy_bin = uv_dir / "pypy-3.10-linux-x86_64-gnu" / "bin" + pypy_bin.mkdir(parents=True) + pypy_exe = pypy_bin / "pypy" + pypy_exe.touch() + Path(str(pypy_bin / "python")).symlink_to(pypy_exe) + + mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) + list(iter_interpreters(cache=session_cache)) + + called_paths = {call.args[0] for call in mock_from_exe.call_args_list} + assert str(pypy_bin / "python") in called_paths + assert str(pypy_exe) not in called_paths From 44f78afd81b3abad8513173667ceae29cc8f198f Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 5 May 2026 13:41:30 +0000 Subject: [PATCH 2/7] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/explanation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/explanation.rst b/docs/explanation.rst index cfbf8f4..f03fe99 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -87,7 +87,7 @@ compatibility with tools that have always read "no implementation in the spec" a :func:`~python_discovery.iter_interpreters` with no spec broadens the search to every name in :data:`~python_discovery.KNOWN_IMPLEMENTATIONS` -- otherwise an "all interpreters" call would silently miss every PyPy and GraalPy on the system. When you pass a spec to :func:`~python_discovery.iter_interpreters`, it falls back -to the same narrow regex as :func:`~python_discovery.get_interpreter`, so behaviour is consistent across the two +to the same narrow regex as :func:`~python_discovery.get_interpreter`, so behavior is consistent across the two APIs whenever a spec is given. **Deduplication.** :func:`~python_discovery.get_interpreter` deduplicates per call so it does not interrogate the From e16d81973becbb610da5cae7a27065d1c3e182e1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 07:03:14 -0700 Subject: [PATCH 3/7] =?UTF-8?q?=F0=9F=94=A7=20chore(discovery):=20satisfy?= =?UTF-8?q?=20lint=20after=20iter=5Finterpreters?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The original `iter_interpreters` body crossed the C901 complexity threshold once all the candidate-skip branches were inlined. Splitting the per-spec loop into `_iter_for_spec` keeps each function under the limit and reads better as a pipeline. Also picks up the codespell `behaviour → behavior` rewrite from the en-GB to en-US dictionary. --- src/python_discovery/_discovery.py | 61 +++++++++++++++++------------- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/src/python_discovery/_discovery.py b/src/python_discovery/_discovery.py index f676d2e..c6ff11a 100644 --- a/src/python_discovery/_discovery.py +++ b/src/python_discovery/_discovery.py @@ -74,39 +74,48 @@ def iter_interpreters( :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. :param predicate: optional filter applied after the spec match; return ``True`` to include the interpreter. """ - env_map = os.environ if env is None else env - try_first_tuple = tuple(try_first_with or ()) if key is None: keys: tuple[str | None, ...] = (None,) elif isinstance(key, str): keys = (key,) else: keys = tuple(key) - seen_real_paths: set[str] = set() + first_with = tuple(try_first_with or ()) + env_map = os.environ if env is None else env + seen: set[str] = set() for spec_str in keys: - if spec_str is None: - spec = PythonSpec("", None, None, None, None, None, None) - wide = True - else: - spec = PythonSpec.from_string_spec(spec_str) - wide = False - for interpreter, impl_must_match in propose_interpreters( - spec, try_first_tuple, cache, env_map, all_implementations=wide - ): - if interpreter is None: - continue - anchor = interpreter.system_executable or interpreter.executable - if anchor is None: - continue - real_path = os.path.realpath(anchor) - if real_path in seen_real_paths: - continue - if not interpreter.satisfies(spec, impl_must_match=impl_must_match): - continue - if predicate is not None and not predicate(interpreter): - continue - seen_real_paths.add(real_path) - yield interpreter + yield from _iter_for_spec(spec_str, first_with, cache, env_map, predicate, seen) + + +def _iter_for_spec( # noqa: PLR0913, PLR0917 + spec_str: str | None, + try_first_with: tuple[str, ...], + cache: PyInfoCache | None, + env: Mapping[str, str], + predicate: Callable[[PythonInfo], bool] | None, + seen: set[str], +) -> Iterator[PythonInfo]: + if spec_str is None: + spec = PythonSpec("", None, None, None, None, None, None) + wide = True + else: + spec = PythonSpec.from_string_spec(spec_str) + wide = False + for interpreter, impl_must_match in propose_interpreters( + spec, try_first_with, cache, env, all_implementations=wide + ): + if interpreter is None: + continue + if (anchor := interpreter.system_executable or interpreter.executable) is None: + continue + if (real_path := os.path.realpath(anchor)) in seen: + continue + if not interpreter.satisfies(spec, impl_must_match=impl_must_match): + continue + if predicate is not None and not predicate(interpreter): + continue + seen.add(real_path) + yield interpreter def _find_interpreter( From d630420bf8c32717871508a9fdb4810ee133b056 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 07:12:38 -0700 Subject: [PATCH 4/7] =?UTF-8?q?=F0=9F=90=9B=20fix(discovery):=20glob=20uv-?= =?UTF-8?q?managed=20Pythons=20on=20Windows?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Globbing `*/bin/python` against uv's install root only finds anything on Unix. On Windows uv places the CPython binary at `/python.exe` directly and PyPy at `/pypy..exe`, neither under a `bin/` subdirectory. GraalPy is the lone exception: uv keeps it at `bin/graalpy.exe` on every OS. Pattern list now matches uv's actual layout per implementation. Globs that do not match anything are essentially free, so the same list runs on every host and there is no platform branching to keep in sync. A summary of the layout has been added to the explanation docs so future contributors do not have to rediscover this from uv's source. --- docs/explanation.rst | 30 +++++++++++++ src/python_discovery/_discovery.py | 6 +-- tests/test_iter_interpreters.py | 70 ++++++++++++++++++++++++++++-- 3 files changed, 99 insertions(+), 7 deletions(-) diff --git a/docs/explanation.rst b/docs/explanation.rst index f03fe99..f384c2e 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -65,6 +65,36 @@ detects these shims and resolves them to the actual binary. `mise `_ and `asdf `_ work similarly, using the ``MISE_DATA_DIR`` and ``ASDF_DATA_DIR`` environment variables to locate their installations. +How uv-managed Pythons are discovered +--------------------------------------- + +`uv `_ installs Python interpreters under a single root directory (configurable via +``UV_PYTHON_INSTALL_DIR``, otherwise defaulting under ``XDG_DATA_HOME`` or the platform user-data path). Each +install lives in its own subdirectory, but the actual binary location varies by OS and implementation: + +.. list-table:: + :header-rows: 1 + :widths: 25 35 40 + + * - Implementation + - Unix layout + - Windows layout + * - CPython + - ``//bin/python`` + - ``//python.exe`` + * - PyPy + - ``//bin/pypy*`` + - ``//pypy*.exe`` + * - GraalPy + - ``//bin/graalpy`` + - ``//bin/graalpy.exe`` + +GraalPy keeps its ``bin/`` segment on Windows (an upstream choice in uv); PyPy and CPython do not. python-discovery +globs all of these patterns regardless of the host OS, because globs that do not match anything are essentially +free, and the cross-platform list is short. Symlinked aliases inside an install (``bin/python``, +``bin/python3``, ``bin/python3.14`` all pointing at the same real file) are deduplicated by resolved path before +the subprocess interrogation, so each install is interrogated once. + Selecting one interpreter vs. enumerating all of them ------------------------------------------------------- diff --git a/src/python_discovery/_discovery.py b/src/python_discovery/_discovery.py index c6ff11a..0f665e7 100644 --- a/src/python_discovery/_discovery.py +++ b/src/python_discovery/_discovery.py @@ -11,7 +11,7 @@ from ._compat import fs_path_id from ._py_info import PythonInfo -from ._py_spec import KNOWN_IMPLEMENTATIONS, PythonSpec +from ._py_spec import PythonSpec if TYPE_CHECKING: from collections.abc import Callable, Generator, Iterable, Iterator, Mapping, Sequence @@ -272,9 +272,9 @@ def _propose_from_uv( else: uv_python_path = user_data_path("uv") / "python" - patterns: list[str] = ["*/bin/python"] + patterns: list[str] = ["*/bin/python", "*/python.exe"] if all_implementations: - patterns.extend(f"*/bin/{impl}" for impl in KNOWN_IMPLEMENTATIONS if impl != "python") + patterns.extend(("*/bin/pypy*", "*/bin/graalpy", "*/pypy*.exe", "*/bin/graalpy.exe")) seen_uv_paths: set[str] = set() for pattern in patterns: for exe_path in uv_python_path.glob(pattern): diff --git a/tests/test_iter_interpreters.py b/tests/test_iter_interpreters.py index 025ac45..598dc2a 100644 --- a/tests/test_iter_interpreters.py +++ b/tests/test_iter_interpreters.py @@ -140,13 +140,75 @@ def test_iter_interpreters_uv_wide_pattern_finds_pypy( pypy_bin = uv_dir / "pypy-3.10-linux-x86_64-gnu" / "bin" pypy_bin.mkdir(parents=True) - pypy_exe = pypy_bin / "pypy" + pypy_exe = pypy_bin / "pypy3.10" pypy_exe.touch() Path(str(pypy_bin / "python")).symlink_to(pypy_exe) + Path(str(pypy_bin / "pypy")).symlink_to(pypy_exe) mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) list(iter_interpreters(cache=session_cache)) - called_paths = {call.args[0] for call in mock_from_exe.call_args_list} - assert str(pypy_bin / "python") in called_paths - assert str(pypy_exe) not in called_paths + interrogated = {call.args[0] for call in mock_from_exe.call_args_list} + assert any(p.endswith(("pypy3.10", "pypy", "python")) for p in interrogated) + resolved = {os.path.realpath(p) for p in interrogated} + assert resolved == {os.path.realpath(pypy_exe)} + + +def test_iter_interpreters_uv_finds_windows_layout( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + monkeypatch.setenv("PATH", "") + uv_dir = tmp_path / "uv-python" + uv_dir.mkdir() + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) + + cpython_root = uv_dir / "cpython-3.14.4-windows-x86_64-none" + cpython_root.mkdir() + cpython_exe = cpython_root / "python.exe" + cpython_exe.touch() + + pypy_root = uv_dir / "pypy-3.11.15-windows-x86_64-none" + pypy_root.mkdir() + pypy_exe = pypy_root / "pypy3.11.exe" + pypy_exe.touch() + + graalpy_bin = uv_dir / "graalpy-3.11-windows-x86_64-none" / "bin" + graalpy_bin.mkdir(parents=True) + graalpy_exe = graalpy_bin / "graalpy.exe" + graalpy_exe.touch() + + mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) + list(iter_interpreters(cache=session_cache)) + + interrogated = {call.args[0] for call in mock_from_exe.call_args_list} + assert str(cpython_exe) in interrogated + assert str(pypy_exe) in interrogated + assert str(graalpy_exe) in interrogated + + +def test_iter_interpreters_uv_finds_graalpy_unix( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + monkeypatch.setenv("PATH", "") + uv_dir = tmp_path / "uv-python" + uv_dir.mkdir() + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) + + graalpy_bin = uv_dir / "graalpy-24.1.1-linux-x86_64-gnu" / "bin" + graalpy_bin.mkdir(parents=True) + graalpy_exe = graalpy_bin / "graalpy" + graalpy_exe.touch() + + mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) + list(iter_interpreters(cache=session_cache)) + + interrogated = {call.args[0] for call in mock_from_exe.call_args_list} + assert str(graalpy_exe) in interrogated From 9bbb7a6999fe8308b79c624a6dbc282acaf236c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 07:17:58 -0700 Subject: [PATCH 5/7] =?UTF-8?q?=F0=9F=A7=AA=20test(iter-interpreters):=20p?= =?UTF-8?q?arametrize,=20hoist=20common=20uv=20setup?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three near-identical uv-layout tests (Unix PyPy, Unix GraalPy, Windows multi-impl) collapse into one parametrized test with explicit IDs. Predicate and "filters to empty" cases get the same treatment. A new `uv_dir` fixture isolates the uv environment by pointing `UV_PYTHON_INSTALL_DIR` at a fresh empty tmp directory, so tests no longer scan whatever the host has under `~/.local/share/uv`. PLR0917 joins PLR0913 in the test-file lint ignores: pytest fixtures naturally bring many parameters and that is not a code-smell. Adds the bugfix changelog fragment for the Windows uv discovery fix landed in the previous commit. --- docs/changelog/65.bugfix.rst | 3 + pyproject.toml | 1 + tests/test_iter_interpreters.py | 225 +++++++++++++++----------------- 3 files changed, 108 insertions(+), 121 deletions(-) create mode 100644 docs/changelog/65.bugfix.rst diff --git a/docs/changelog/65.bugfix.rst b/docs/changelog/65.bugfix.rst new file mode 100644 index 0000000..cde8158 --- /dev/null +++ b/docs/changelog/65.bugfix.rst @@ -0,0 +1,3 @@ +discover uv-managed Pythons on Windows. Previously the glob assumed Unix layout (``//bin/python``) and +silently found nothing on Windows, where uv places ``python.exe`` directly under the install root - by +:user:`gaborbernat`. diff --git a/pyproject.toml b/pyproject.toml index eaf4475..4332a6c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -105,6 +105,7 @@ lint.per-file-ignores."tests/**/*.py" = [ "PLC0415", # imports inside test functions (conditional on mocking) "PLC2701", # private imports needed to test internal APIs "PLR0913", # too many arguments (pytest fixtures) + "PLR0917", # too many positional arguments (pytest fixtures) "PLR2004", # Magic value used in comparison "S101", # asserts allowed in tests "S404", # subprocess import diff --git a/tests/test_iter_interpreters.py b/tests/test_iter_interpreters.py index 598dc2a..e2ef514 100644 --- a/tests/test_iter_interpreters.py +++ b/tests/test_iter_interpreters.py @@ -6,54 +6,82 @@ from typing import TYPE_CHECKING from uuid import uuid4 +import pytest + from python_discovery import DiskCache, PythonInfo, iter_interpreters from python_discovery._discovery import IS_WIN if TYPE_CHECKING: - from collections.abc import Iterator + from collections.abc import Callable, Iterator - import pytest from pytest_mock import MockerFixture -def test_iter_interpreters_with_abs_path_yields_one(session_cache: DiskCache) -> None: - results = list(iter_interpreters(sys.executable, cache=session_cache)) - assert len(results) == 1 - assert results[0].executable == sys.executable +@pytest.fixture +def uv_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Empty uv install root with environment variables pointing at it.""" + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + root = tmp_path / "uv-python" + root.mkdir() + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(root)) + return root -def test_iter_interpreters_predicate_filters_all(session_cache: DiskCache) -> None: - results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=lambda _: False)) - assert results == [] +def test_iter_interpreters_returns_iterator(session_cache: DiskCache) -> None: + result: Iterator[PythonInfo] = iter_interpreters(sys.executable, cache=session_cache) + assert iter(result) is result -def test_iter_interpreters_predicate_keeps_all(session_cache: DiskCache) -> None: - results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=lambda _: True)) +def test_iter_interpreters_with_abs_path_yields_one(session_cache: DiskCache) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache)) assert len(results) == 1 + assert results[0].executable == sys.executable -def test_iter_interpreters_no_match(session_cache: DiskCache) -> None: - results = list(iter_interpreters(uuid4().hex, cache=session_cache)) - assert results == [] - - -def test_iter_interpreters_satisfies_filter_drops_all(session_cache: DiskCache) -> None: - results = list(iter_interpreters(">=999", cache=session_cache)) - assert results == [] +def test_iter_interpreters_sequence_dedups_across_keys(session_cache: DiskCache) -> None: + results = list(iter_interpreters([sys.executable, sys.executable], cache=session_cache)) + assert len(results) == 1 -def test_iter_interpreters_no_key_includes_running_interpreter( - monkeypatch: pytest.MonkeyPatch, - tmp_path: Path, +@pytest.mark.parametrize( + ("predicate", "expected_count"), + [ + pytest.param(None, 1, id="none-is-noop"), + pytest.param(lambda _: True, 1, id="accepts-all"), + pytest.param(lambda _: False, 0, id="rejects-all"), + ], +) +def test_iter_interpreters_predicate( session_cache: DiskCache, + predicate: Callable[[PythonInfo], bool] | None, + expected_count: int, ) -> None: - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) - monkeypatch.delenv("XDG_DATA_HOME", raising=False) + results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=predicate)) + assert len(results) == expected_count + + +@pytest.mark.parametrize( + "key_factory", + [ + pytest.param(lambda: uuid4().hex, id="filename-cannot-match"), + pytest.param(lambda: ">=999", id="version-cannot-match"), + ], +) +def test_iter_interpreters_filters_to_empty( + session_cache: DiskCache, + key_factory: Callable[[], str], +) -> None: + assert list(iter_interpreters(key_factory(), cache=session_cache)) == [] + + +@pytest.mark.usefixtures("uv_dir") +def test_iter_interpreters_no_key_includes_running_interpreter(session_cache: DiskCache) -> None: real_self = os.path.realpath(sys.executable) results = list(iter_interpreters(cache=session_cache)) assert any(os.path.realpath(info.executable) == real_self for info in results) +@pytest.mark.usefixtures("uv_dir") def test_iter_interpreters_dedups_symlinks( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, @@ -65,8 +93,6 @@ def test_iter_interpreters_dedups_symlinks( pyvenv_cfg = Path(sys.executable).parents[1] / "pyvenv.cfg" if pyvenv_cfg.exists(): # pragma: no branch (tmp_path / pyvenv_cfg.name).write_bytes(pyvenv_cfg.read_bytes()) - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) - monkeypatch.delenv("XDG_DATA_HOME", raising=False) monkeypatch.setenv("PATH", os.pathsep.join([str(tmp_path), os.environ.get("PATH", "")])) real = os.path.realpath(sys.executable) @@ -74,29 +100,18 @@ def test_iter_interpreters_dedups_symlinks( assert len(matches) == 1 -def test_iter_interpreters_sequence_dedups_across_keys(session_cache: DiskCache) -> None: - results = list(iter_interpreters([sys.executable, sys.executable], cache=session_cache)) - assert len(results) == 1 - - +@pytest.mark.usefixtures("uv_dir") def test_iter_interpreters_try_first_with_yields_first( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, session_cache: DiskCache, ) -> None: monkeypatch.setenv("PATH", str(tmp_path)) - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(tmp_path / "no-such-uv")) - monkeypatch.delenv("XDG_DATA_HOME", raising=False) results = list(iter_interpreters(try_first_with=[sys.executable], cache=session_cache)) assert results assert os.path.realpath(results[0].executable) == os.path.realpath(sys.executable) -def test_iter_interpreters_returns_iterator(session_cache: DiskCache) -> None: - result: Iterator[PythonInfo] = iter_interpreters(sys.executable, cache=session_cache) - assert iter(result) is result - - def test_iter_interpreters_skips_none_from_propose( session_cache: DiskCache, mocker: MockerFixture, @@ -107,8 +122,7 @@ def test_iter_interpreters_skips_none_from_propose( return_value=iter([(None, True), (real_info, True)]), ) results = list(iter_interpreters(cache=session_cache)) - assert len(results) == 1 - assert results[0] is real_info + assert results == [real_info] def test_iter_interpreters_skips_when_no_executable( @@ -122,93 +136,62 @@ def test_iter_interpreters_skips_when_no_executable( "python_discovery._discovery.propose_interpreters", return_value=iter([(bogus, True)]), ) - results = list(iter_interpreters(cache=session_cache)) - assert results == [] - - -def test_iter_interpreters_uv_wide_pattern_finds_pypy( - monkeypatch: pytest.MonkeyPatch, - tmp_path: Path, - session_cache: DiskCache, - mocker: MockerFixture, -) -> None: - monkeypatch.delenv("XDG_DATA_HOME", raising=False) - monkeypatch.setenv("PATH", "") - uv_dir = tmp_path / "uv-python" - uv_dir.mkdir() - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) - - pypy_bin = uv_dir / "pypy-3.10-linux-x86_64-gnu" / "bin" - pypy_bin.mkdir(parents=True) - pypy_exe = pypy_bin / "pypy3.10" - pypy_exe.touch() - Path(str(pypy_bin / "python")).symlink_to(pypy_exe) - Path(str(pypy_bin / "pypy")).symlink_to(pypy_exe) - - mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) - list(iter_interpreters(cache=session_cache)) - - interrogated = {call.args[0] for call in mock_from_exe.call_args_list} - assert any(p.endswith(("pypy3.10", "pypy", "python")) for p in interrogated) - resolved = {os.path.realpath(p) for p in interrogated} - assert resolved == {os.path.realpath(pypy_exe)} - - -def test_iter_interpreters_uv_finds_windows_layout( - monkeypatch: pytest.MonkeyPatch, - tmp_path: Path, + assert list(iter_interpreters(cache=session_cache)) == [] + + +@pytest.mark.parametrize( + ("real_files", "symlinks", "expected_real"), + [ + pytest.param( + ("pypy-3.10-linux-x86_64-gnu/bin/pypy3.10",), + ( + ("pypy-3.10-linux-x86_64-gnu/bin/python", "pypy-3.10-linux-x86_64-gnu/bin/pypy3.10"), + ("pypy-3.10-linux-x86_64-gnu/bin/pypy", "pypy-3.10-linux-x86_64-gnu/bin/pypy3.10"), + ), + ("pypy-3.10-linux-x86_64-gnu/bin/pypy3.10",), + id="pypy-unix-with-python-symlink", + ), + pytest.param( + ("graalpy-24.1.1-linux-x86_64-gnu/bin/graalpy",), + (), + ("graalpy-24.1.1-linux-x86_64-gnu/bin/graalpy",), + id="graalpy-unix", + ), + pytest.param( + ( + "cpython-3.14.4-windows-x86_64-none/python.exe", + "pypy-3.11.15-windows-x86_64-none/pypy3.11.exe", + "graalpy-3.11-windows-x86_64-none/bin/graalpy.exe", + ), + (), + ( + "cpython-3.14.4-windows-x86_64-none/python.exe", + "pypy-3.11.15-windows-x86_64-none/pypy3.11.exe", + "graalpy-3.11-windows-x86_64-none/bin/graalpy.exe", + ), + id="windows-all-impls", + ), + ], +) +def test_iter_interpreters_uv_layout( + uv_dir: Path, session_cache: DiskCache, - mocker: MockerFixture, -) -> None: - monkeypatch.delenv("XDG_DATA_HOME", raising=False) - monkeypatch.setenv("PATH", "") - uv_dir = tmp_path / "uv-python" - uv_dir.mkdir() - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) - - cpython_root = uv_dir / "cpython-3.14.4-windows-x86_64-none" - cpython_root.mkdir() - cpython_exe = cpython_root / "python.exe" - cpython_exe.touch() - - pypy_root = uv_dir / "pypy-3.11.15-windows-x86_64-none" - pypy_root.mkdir() - pypy_exe = pypy_root / "pypy3.11.exe" - pypy_exe.touch() - - graalpy_bin = uv_dir / "graalpy-3.11-windows-x86_64-none" / "bin" - graalpy_bin.mkdir(parents=True) - graalpy_exe = graalpy_bin / "graalpy.exe" - graalpy_exe.touch() - - mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) - list(iter_interpreters(cache=session_cache)) - - interrogated = {call.args[0] for call in mock_from_exe.call_args_list} - assert str(cpython_exe) in interrogated - assert str(pypy_exe) in interrogated - assert str(graalpy_exe) in interrogated - - -def test_iter_interpreters_uv_finds_graalpy_unix( monkeypatch: pytest.MonkeyPatch, - tmp_path: Path, - session_cache: DiskCache, mocker: MockerFixture, + real_files: tuple[str, ...], + symlinks: tuple[tuple[str, str], ...], + expected_real: tuple[str, ...], ) -> None: - monkeypatch.delenv("XDG_DATA_HOME", raising=False) monkeypatch.setenv("PATH", "") - uv_dir = tmp_path / "uv-python" - uv_dir.mkdir() - monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(uv_dir)) - - graalpy_bin = uv_dir / "graalpy-24.1.1-linux-x86_64-gnu" / "bin" - graalpy_bin.mkdir(parents=True) - graalpy_exe = graalpy_bin / "graalpy" - graalpy_exe.touch() + for rel in real_files: + path = uv_dir / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.touch() + for link_rel, target_rel in symlinks: + Path(str(uv_dir / link_rel)).symlink_to(uv_dir / target_rel) mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) list(iter_interpreters(cache=session_cache)) - interrogated = {call.args[0] for call in mock_from_exe.call_args_list} - assert str(graalpy_exe) in interrogated + interrogated_real = {os.path.realpath(call.args[0]) for call in mock_from_exe.call_args_list} + assert interrogated_real == {os.path.realpath(uv_dir / r) for r in expected_real} From 2975707c0481c56e4bc5ddee20be3e4e4a7d69c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 07:27:47 -0700 Subject: [PATCH 6/7] =?UTF-8?q?=F0=9F=A7=AA=20test(iter-interpreters):=20d?= =?UTF-8?q?rop=20pragmas,=20link=20to=20packaging=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces `# pragma: no branch` on the uv interrogation yield with a real test that exercises the success path. The test stubs `from_exe` to return a `MagicMock(spec=PythonInfo)` with a unique `system_executable` so the result survives realpath dedup and shows up in the yielded list. The pyvenv.cfg copy in the symlink-dedup test is now wrapped in `contextlib.suppress(FileNotFoundError)` instead of an existence check, which avoids the conditional that needed pragma'ing while preserving the behavior on machines that do or do not run from inside a venv. PEP 440 references in user-facing docs and docstrings now link to `packaging.python.org/en/latest/specifications/version-specifiers/`, which is the canonical, maintained source of the spec - PEPs are historical and the PyPA spec page is what readers should land on. PEP 514 stays as a PEP link since the Windows registry spec is not mirrored on packaging.python.org. --- README.md | 3 ++- docs/explanation.rst | 13 ++++++++----- docs/tutorial/getting-started.rst | 3 ++- src/python_discovery/_discovery.py | 7 ++++--- src/python_discovery/_py_spec.py | 6 ++++-- tests/test_iter_interpreters.py | 23 ++++++++++++++++++++++- 6 files changed, 42 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 5710f84..2658525 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,8 @@ The `get_interpreter()` function accepts various specification formats: - Absolute path: `/usr/bin/python3.12` - Version: `3.12` - Implementation prefix: `cpython3.12` -- PEP 440 specifier: `>=3.10`, `>=3.11,<3.13` +- [Version specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/): `>=3.10`, + `>=3.11,<3.13` ## Documentation diff --git a/docs/explanation.rst b/docs/explanation.rst index f384c2e..8c764bd 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -128,7 +128,7 @@ symlinks to its base interpreter, collapse to a single yield. The semantic is "o which is what callers building choosers or version-range pickers usually want. **Iteration order.** Yields come back in *priority order*: ``try_first_with`` first, then the running interpreter, -then PEP 514 entries on Windows, then PATH left-to-right, then UV-managed installs. This matches what +then :pep:`514` entries on Windows, then PATH left-to-right, then UV-managed installs. This matches what :func:`~python_discovery.get_interpreter` would have returned at each step. If your ordering differs (newest version first, smallest install root, etc.), wrap the call in :func:`sorted` -- the API deliberately does not include a ``sort_by`` parameter because keeping discovery order preserves the priority signal for callers who @@ -234,9 +234,12 @@ A spec string follows the pattern ``[impl][version][t][-arch][-machine]``. Every * - ``/usr/bin/python3`` - Absolute path, used directly (no search) * - ``>=3.11,<3.13`` - - :pep:`440` version specifier (any Python in range) + - `Version specifier `_ + (any Python in range) * - ``cpython>=3.11`` - - :pep:`440` specifier restricted to CPython + - `Version specifier `_ + restricted to CPython -:pep:`440` specifiers (``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple -specifiers can be comma-separated, for example ``>=3.11,<3.13``. +`Version specifiers `_ +(``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple specifiers can be comma-separated, +for example ``>=3.11,<3.13``. diff --git a/docs/tutorial/getting-started.rst b/docs/tutorial/getting-started.rst index 5f1c3ca..1c10bc4 100644 --- a/docs/tutorial/getting-started.rst +++ b/docs/tutorial/getting-started.rst @@ -174,7 +174,8 @@ Common examples: * - ``/usr/bin/python3`` - An absolute path, used directly without searching * - ``>=3.11,<3.13`` - - Any Python in the 3.11--3.12 range (:pep:`440` syntax) + - Any Python in the 3.11--3.12 range + (`version specifier `_ syntax) See the :doc:`full spec reference ` for all options. diff --git a/src/python_discovery/_discovery.py b/src/python_discovery/_discovery.py index 0f665e7..67df15b 100644 --- a/src/python_discovery/_discovery.py +++ b/src/python_discovery/_discovery.py @@ -36,7 +36,9 @@ def get_interpreter( the optional *predicate*. :param key: interpreter specification string(s) — an absolute path, a version (``3.12``), an implementation prefix - (``cpython3.12``), or a PEP 440 specifier (``>=3.10``). When a sequence is given each entry is tried in order. + (``cpython3.12``), or a + `version specifier `_ + (``>=3.10``). When a sequence is given each entry is tried in order. :param try_first_with: executables to probe before the normal discovery search. :param cache: interpreter metadata cache; when ``None`` results are not cached. :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. @@ -282,8 +284,7 @@ def _propose_from_uv( if resolved in seen_uv_paths: continue seen_uv_paths.add(resolved) - interpreter = PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env) - if interpreter is not None: # pragma: no branch + if interpreter := PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env): yield interpreter, True diff --git a/src/python_discovery/_py_spec.py b/src/python_discovery/_py_spec.py index 8c89442..e3cf590 100644 --- a/src/python_discovery/_py_spec.py +++ b/src/python_discovery/_py_spec.py @@ -113,7 +113,9 @@ class PythonSpec: :param path: filesystem path to a specific interpreter, or ``None``. :param free_threaded: whether a free-threaded build is required, or ``None`` for any. :param machine: required ISA (e.g. ``"arm64"``), or ``None`` for any. - :param version_specifier: PEP 440 version constraints, or ``None``. + :param version_specifier: + `version specifier `_ + constraints, or ``None``. """ def __init__( # noqa: PLR0913, PLR0917 @@ -147,7 +149,7 @@ def from_string_spec(cls, string_spec: str) -> PythonSpec: Parse a string specification into a :class:`PythonSpec`. :param string_spec: an interpreter spec — an absolute path, a version string, an implementation prefix, - or a PEP 440 specifier. + or a `version specifier `_. """ if pathlib.Path(string_spec).is_absolute(): return cls(string_spec, None, None, None, None, None, string_spec) diff --git a/tests/test_iter_interpreters.py b/tests/test_iter_interpreters.py index e2ef514..5f50a8b 100644 --- a/tests/test_iter_interpreters.py +++ b/tests/test_iter_interpreters.py @@ -1,5 +1,6 @@ from __future__ import annotations +import contextlib import os import sys from pathlib import Path @@ -91,7 +92,7 @@ def test_iter_interpreters_dedups_symlinks( for name in ("python3", "python3.99"): Path(str(tmp_path / f"{name}{suffix}")).symlink_to(sys.executable) pyvenv_cfg = Path(sys.executable).parents[1] / "pyvenv.cfg" - if pyvenv_cfg.exists(): # pragma: no branch + with contextlib.suppress(FileNotFoundError): (tmp_path / pyvenv_cfg.name).write_bytes(pyvenv_cfg.read_bytes()) monkeypatch.setenv("PATH", os.pathsep.join([str(tmp_path), os.environ.get("PATH", "")])) @@ -139,6 +140,26 @@ def test_iter_interpreters_skips_when_no_executable( assert list(iter_interpreters(cache=session_cache)) == [] +def test_iter_interpreters_uv_yields_when_interrogation_succeeds( + uv_dir: Path, + session_cache: DiskCache, + monkeypatch: pytest.MonkeyPatch, + mocker: MockerFixture, +) -> None: + monkeypatch.setenv("PATH", "") + bin_path = uv_dir / "cpython-3.99-fake/bin" + bin_path.mkdir(parents=True) + (bin_path / "python").touch() + + fake_info = mocker.MagicMock(spec=PythonInfo) + fake_info.system_executable = str(uv_dir / "fake-unique" / "python") + fake_info.executable = fake_info.system_executable + fake_info.satisfies.return_value = True + mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=fake_info) + + assert fake_info in list(iter_interpreters(cache=session_cache)) + + @pytest.mark.parametrize( ("real_files", "symlinks", "expected_real"), [ From 5bb5267d0e56a0d2945910ea873f12938149a813 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bern=C3=A1t=20G=C3=A1bor?= Date: Tue, 5 May 2026 07:31:27 -0700 Subject: [PATCH 7/7] =?UTF-8?q?=F0=9F=93=9D=20docs(explanation):=20add=20c?= =?UTF-8?q?olored=20uv=20discovery=20flow=20diagram?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pairs the per-OS layout table with a mermaid flowchart that shows which glob pattern fires under narrow vs wide mode, where realpath dedup sits in the pipeline, and where subprocess interrogation happens. Colors match the existing diagrams in the file: blue for inputs, orange for decisions and dedup, purple for the wide-mode-only branches, green for the final action. --- docs/explanation.rst | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/explanation.rst b/docs/explanation.rst index 8c764bd..e35300c 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -89,6 +89,37 @@ install lives in its own subdirectory, but the actual binary location varies by - ``//bin/graalpy`` - ``//bin/graalpy.exe`` +.. mermaid:: + + flowchart LR + Call(["iter_interpreters(key)"]) --> Mode{"key is None?"} + Mode -->|"narrow"| N1["*/bin/python"] + Mode -->|"narrow"| N2["*/python.exe"] + Mode -->|"wide"| W1["*/bin/pypy*"] + Mode -->|"wide"| W2["*/bin/graalpy"] + Mode -->|"wide"| W3["*/pypy*.exe"] + Mode -->|"wide"| W4["*/bin/graalpy.exe"] + + N1 --> Dedup[/"realpath dedup"/] + N2 --> Dedup + W1 --> Dedup + W2 --> Dedup + W3 --> Dedup + W4 --> Dedup + + Dedup --> Interrogate(["subprocess interrogation"]) + + style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff + style Mode fill:#d9904a,stroke:#8f5f2a,color:#fff + style N1 fill:#3a7fc2,stroke:#1f4d7a,color:#fff + style N2 fill:#3a7fc2,stroke:#1f4d7a,color:#fff + style W1 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W2 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W3 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W4 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style Dedup fill:#c2873a,stroke:#7a4c1f,color:#fff + style Interrogate fill:#4a9f4a,stroke:#2a6f2a,color:#fff + GraalPy keeps its ``bin/`` segment on Windows (an upstream choice in uv); PyPy and CPython do not. python-discovery globs all of these patterns regardless of the host OS, because globs that do not match anything are essentially free, and the cross-platform list is short. Symlinked aliases inside an install (``bin/python``,