diff --git a/README.md b/README.md index 5710f84..2658525 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,8 @@ The `get_interpreter()` function accepts various specification formats: - Absolute path: `/usr/bin/python3.12` - Version: `3.12` - Implementation prefix: `cpython3.12` -- PEP 440 specifier: `>=3.10`, `>=3.11,<3.13` +- [Version specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/): `>=3.10`, + `>=3.11,<3.13` ## Documentation diff --git a/docs/changelog/65.bugfix.rst b/docs/changelog/65.bugfix.rst new file mode 100644 index 0000000..cde8158 --- /dev/null +++ b/docs/changelog/65.bugfix.rst @@ -0,0 +1,3 @@ +discover uv-managed Pythons on Windows. Previously the glob assumed Unix layout (``//bin/python``) and +silently found nothing on Windows, where uv places ``python.exe`` directly under the install root - by +:user:`gaborbernat`. diff --git a/docs/changelog/65.feature.rst b/docs/changelog/65.feature.rst new file mode 100644 index 0000000..412641f --- /dev/null +++ b/docs/changelog/65.feature.rst @@ -0,0 +1,3 @@ +add :func:`~python_discovery.iter_interpreters` for enumerating every discovered interpreter, with PATH and +UV-install support for non-CPython implementations listed in :data:`~python_discovery.KNOWN_IMPLEMENTATIONS` +- by :user:`gaborbernat`. diff --git a/docs/explanation.rst b/docs/explanation.rst index 2d1bc22..e35300c 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -65,6 +65,106 @@ detects these shims and resolves them to the actual binary. `mise `_ and `asdf `_ work similarly, using the ``MISE_DATA_DIR`` and ``ASDF_DATA_DIR`` environment variables to locate their installations. +How uv-managed Pythons are discovered +--------------------------------------- + +`uv `_ installs Python interpreters under a single root directory (configurable via +``UV_PYTHON_INSTALL_DIR``, otherwise defaulting under ``XDG_DATA_HOME`` or the platform user-data path). Each +install lives in its own subdirectory, but the actual binary location varies by OS and implementation: + +.. list-table:: + :header-rows: 1 + :widths: 25 35 40 + + * - Implementation + - Unix layout + - Windows layout + * - CPython + - ``//bin/python`` + - ``//python.exe`` + * - PyPy + - ``//bin/pypy*`` + - ``//pypy*.exe`` + * - GraalPy + - ``//bin/graalpy`` + - ``//bin/graalpy.exe`` + +.. mermaid:: + + flowchart LR + Call(["iter_interpreters(key)"]) --> Mode{"key is None?"} + Mode -->|"narrow"| N1["*/bin/python"] + Mode -->|"narrow"| N2["*/python.exe"] + Mode -->|"wide"| W1["*/bin/pypy*"] + Mode -->|"wide"| W2["*/bin/graalpy"] + Mode -->|"wide"| W3["*/pypy*.exe"] + Mode -->|"wide"| W4["*/bin/graalpy.exe"] + + N1 --> Dedup[/"realpath dedup"/] + N2 --> Dedup + W1 --> Dedup + W2 --> Dedup + W3 --> Dedup + W4 --> Dedup + + Dedup --> Interrogate(["subprocess interrogation"]) + + style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff + style Mode fill:#d9904a,stroke:#8f5f2a,color:#fff + style N1 fill:#3a7fc2,stroke:#1f4d7a,color:#fff + style N2 fill:#3a7fc2,stroke:#1f4d7a,color:#fff + style W1 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W2 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W3 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style W4 fill:#9f4ad9,stroke:#5f2a8f,color:#fff + style Dedup fill:#c2873a,stroke:#7a4c1f,color:#fff + style Interrogate fill:#4a9f4a,stroke:#2a6f2a,color:#fff + +GraalPy keeps its ``bin/`` segment on Windows (an upstream choice in uv); PyPy and CPython do not. python-discovery +globs all of these patterns regardless of the host OS, because globs that do not match anything are essentially +free, and the cross-platform list is short. Symlinked aliases inside an install (``bin/python``, +``bin/python3``, ``bin/python3.14`` all pointing at the same real file) are deduplicated by resolved path before +the subprocess interrogation, so each install is interrogated once. + +Selecting one interpreter vs. enumerating all of them +------------------------------------------------------- + +:func:`~python_discovery.get_interpreter` and :func:`~python_discovery.iter_interpreters` walk the same candidate +sources, but they answer different questions and behave differently in three ways. + +.. mermaid:: + + flowchart LR + Sources["candidate sources
(try_first_with → current →
PEP 514 → PATH → uv)"] + Sources --> Get["get_interpreter()
first match wins, returns one"] + Sources --> Iter["iter_interpreters()
yields every match"] + + style Get fill:#4a9f4a,stroke:#2a6f2a,color:#fff + style Iter fill:#4a90d9,stroke:#2a5f8f,color:#fff + +**Implementation coverage on PATH.** :func:`~python_discovery.get_interpreter` matches only ``python*`` filenames on +PATH unless the spec names another implementation explicitly (``pypy3.12``, ``graalpy3.11``). This keeps backwards +compatibility with tools that have always read "no implementation in the spec" as "give me CPython." +:func:`~python_discovery.iter_interpreters` with no spec broadens the search to every name in +:data:`~python_discovery.KNOWN_IMPLEMENTATIONS` -- otherwise an "all interpreters" call would silently miss every +PyPy and GraalPy on the system. When you pass a spec to :func:`~python_discovery.iter_interpreters`, it falls back +to the same narrow regex as :func:`~python_discovery.get_interpreter`, so behavior is consistent across the two +APIs whenever a spec is given. + +**Deduplication.** :func:`~python_discovery.get_interpreter` deduplicates per call so it does not interrogate the +same binary twice while searching, and stops as soon as a match is found. :func:`~python_discovery.iter_interpreters` +deduplicates by the resolved real path of each candidate's ``system_executable`` (falling back to ``executable``). +That means symlinked aliases like ``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv whose ``python`` +symlinks to its base interpreter, collapse to a single yield. The semantic is "one entry per distinct install," +which is what callers building choosers or version-range pickers usually want. + +**Iteration order.** Yields come back in *priority order*: ``try_first_with`` first, then the running interpreter, +then :pep:`514` entries on Windows, then PATH left-to-right, then UV-managed installs. This matches what +:func:`~python_discovery.get_interpreter` would have returned at each step. If your ordering differs (newest +version first, smallest install root, etc.), wrap the call in :func:`sorted` -- the API deliberately does not +include a ``sort_by`` parameter because keeping discovery order preserves the priority signal for callers who +want it. + How caching works ------------------- @@ -165,9 +265,12 @@ A spec string follows the pattern ``[impl][version][t][-arch][-machine]``. Every * - ``/usr/bin/python3`` - Absolute path, used directly (no search) * - ``>=3.11,<3.13`` - - :pep:`440` version specifier (any Python in range) + - `Version specifier `_ + (any Python in range) * - ``cpython>=3.11`` - - :pep:`440` specifier restricted to CPython + - `Version specifier `_ + restricted to CPython -:pep:`440` specifiers (``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple -specifiers can be comma-separated, for example ``>=3.11,<3.13``. +`Version specifiers `_ +(``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple specifiers can be comma-separated, +for example ``>=3.11,<3.13``. diff --git a/docs/how-to/standalone-usage.rst b/docs/how-to/standalone-usage.rst index adf151e..507cdd2 100644 --- a/docs/how-to/standalone-usage.rst +++ b/docs/how-to/standalone-usage.rst @@ -62,6 +62,60 @@ the ``PY_DISCOVERY_TIMEOUT`` environment variable. The timeout value should be a number in seconds. Each interpreter candidate is given this much time to respond. If a timeout occurs, the candidate is skipped and the search continues with the next one. +List every interpreter on the system +-------------------------------------- + +Use :func:`~python_discovery.iter_interpreters` to enumerate every Python python-discovery can find. With no spec +it yields all known implementations (CPython, PyPy, GraalPy -- see +:data:`~python_discovery.KNOWN_IMPLEMENTATIONS`). Pass a spec to filter, exactly like +:func:`~python_discovery.get_interpreter`. + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + + # Every interpreter, no filter + for info in iter_interpreters(cache=cache): + print(info.executable, info.version_str, info.implementation) + + # Every CPython 3.10 or newer, newest first + newest_first = sorted( + iter_interpreters("cpython>=3.10", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + +Enumeration interrogates every candidate as a subprocess on a cold cache. Always pass a +:class:`~python_discovery.DiskCache` if you call this more than once. + +Pick an interpreter from a range, preferring newer +---------------------------------------------------- + +A common need: search a version range and prefer newer interpreters when more than one matches. Sort the result of +:func:`~python_discovery.iter_interpreters` and take the first. + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + + matches = sorted( + iter_interpreters("cpython>=3.10,<3.15", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + info = matches[0] if matches else None + +Use :func:`get_interpreter` instead when you only need the first PATH-priority hit; use the sort-and-take pattern +when *your* ordering differs from PATH order (newest version, smallest install size, preferred install root, etc.). + Read interpreter metadata --------------------------- diff --git a/docs/tutorial/getting-started.rst b/docs/tutorial/getting-started.rst index 696efcf..1c10bc4 100644 --- a/docs/tutorial/getting-started.rst +++ b/docs/tutorial/getting-started.rst @@ -87,6 +87,49 @@ You can pass multiple specs as a list -- the library tries each one in order and result = get_interpreter(["python3.12", "python3.11"], cache=cache) +Listing every interpreter +--------------------------- + +When you need *every* interpreter rather than just the first match -- for example, to show the user a chooser, or +to apply your own ranking -- use :func:`~python_discovery.iter_interpreters`. Pass no arguments to enumerate every +implementation python-discovery knows about, or pass a spec to filter. + +.. mermaid:: + + flowchart TD + Call["iter_interpreters(spec, cache)"] --> Yield["yields PythonInfo"] + Yield --> A["1. try_first_with paths"] + Yield --> B["2. running interpreter"] + Yield --> C["3. PATH (left to right)"] + Yield --> D["4. uv-managed installs"] + + style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff + style Yield fill:#4a9f4a,stroke:#2a6f2a,color:#fff + +.. code-block:: python + + from pathlib import Path + + from python_discovery import DiskCache, iter_interpreters + + cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser()) + for info in iter_interpreters(cache=cache): + print(info.executable, info.version_str, info.implementation) + +The result is an iterator, so :func:`list`, :func:`sorted`, generator expressions and early ``break`` all work as you +would expect. Symlinked aliases (``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv and the base it points at) +collapse to a single entry, so you do not see the same install twice. + +To prefer newer interpreters in a range, sort the result by ``version_info`` after filtering: + +.. code-block:: python + + newest_first = sorted( + iter_interpreters(">=3.10,<3.15", cache=cache), + key=lambda info: info.version_info, + reverse=True, + ) + Writing specs ------------- @@ -131,7 +174,8 @@ Common examples: * - ``/usr/bin/python3`` - An absolute path, used directly without searching * - ``>=3.11,<3.13`` - - Any Python in the 3.11--3.12 range (:pep:`440` syntax) + - Any Python in the 3.11--3.12 range + (`version specifier `_ syntax) See the :doc:`full spec reference ` for all options. diff --git a/pyproject.toml b/pyproject.toml index eaf4475..4332a6c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -105,6 +105,7 @@ lint.per-file-ignores."tests/**/*.py" = [ "PLC0415", # imports inside test functions (conditional on mocking) "PLC2701", # private imports needed to test internal APIs "PLR0913", # too many arguments (pytest fixtures) + "PLR0917", # too many positional arguments (pytest fixtures) "PLR2004", # Magic value used in comparison "S101", # asserts allowed in tests "S404", # subprocess import diff --git a/src/python_discovery/__init__.py b/src/python_discovery/__init__.py index 40307ae..4533386 100644 --- a/src/python_discovery/__init__.py +++ b/src/python_discovery/__init__.py @@ -5,15 +5,16 @@ from importlib.metadata import version from ._cache import ContentStore, DiskCache, PyInfoCache -from ._discovery import get_interpreter +from ._discovery import get_interpreter, iter_interpreters from ._py_info import KNOWN_ARCHITECTURES, PythonInfo, normalize_isa -from ._py_spec import PythonSpec +from ._py_spec import KNOWN_IMPLEMENTATIONS, PythonSpec from ._specifier import SimpleSpecifier, SimpleSpecifierSet, SimpleVersion __version__ = version("python-discovery") __all__ = [ "KNOWN_ARCHITECTURES", + "KNOWN_IMPLEMENTATIONS", "ContentStore", "DiskCache", "PyInfoCache", @@ -24,5 +25,6 @@ "SimpleVersion", "__version__", "get_interpreter", + "iter_interpreters", "normalize_isa", ] diff --git a/src/python_discovery/_discovery.py b/src/python_discovery/_discovery.py index d534598..67df15b 100644 --- a/src/python_discovery/_discovery.py +++ b/src/python_discovery/_discovery.py @@ -14,7 +14,7 @@ from ._py_spec import PythonSpec if TYPE_CHECKING: - from collections.abc import Callable, Generator, Iterable, Mapping, Sequence + from collections.abc import Callable, Generator, Iterable, Iterator, Mapping, Sequence from ._cache import PyInfoCache @@ -36,7 +36,9 @@ def get_interpreter( the optional *predicate*. :param key: interpreter specification string(s) — an absolute path, a version (``3.12``), an implementation prefix - (``cpython3.12``), or a PEP 440 specifier (``>=3.10``). When a sequence is given each entry is tried in order. + (``cpython3.12``), or a + `version specifier `_ + (``>=3.10``). When a sequence is given each entry is tried in order. :param try_first_with: executables to probe before the normal discovery search. :param cache: interpreter metadata cache; when ``None`` results are not cached. :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. @@ -51,6 +53,73 @@ def get_interpreter( return None +def iter_interpreters( + key: str | Sequence[str] | None = None, + try_first_with: Iterable[str] | None = None, + cache: PyInfoCache | None = None, + env: Mapping[str, str] | None = None, + predicate: Callable[[PythonInfo], bool] | None = None, +) -> Iterator[PythonInfo]: + """ + Yield every interpreter on the system that satisfies *key*. + + Iteration order is discovery order: ``try_first_with`` paths first, then the running interpreter, then ``PATH`` + (left to right), then UV-managed installs. Results are deduplicated by the resolved real path of the underlying + system interpreter, so symlinked aliases (``/bin`` vs ``/usr/bin``) and venvs that symlink to a base interpreter + collapse to a single entry. Callers that want a different ordering should sort the result. + + :param key: interpreter specification — same syntax as :func:`get_interpreter`. ``None`` enumerates every Python + implementation python-discovery knows about (see :data:`KNOWN_IMPLEMENTATIONS`). + :param try_first_with: executables to probe before the normal discovery search. + :param cache: interpreter metadata cache; when ``None`` results are not cached. Strongly recommended for + enumeration, which interrogates every candidate as a subprocess on a cold cache. + :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. + :param predicate: optional filter applied after the spec match; return ``True`` to include the interpreter. + """ + if key is None: + keys: tuple[str | None, ...] = (None,) + elif isinstance(key, str): + keys = (key,) + else: + keys = tuple(key) + first_with = tuple(try_first_with or ()) + env_map = os.environ if env is None else env + seen: set[str] = set() + for spec_str in keys: + yield from _iter_for_spec(spec_str, first_with, cache, env_map, predicate, seen) + + +def _iter_for_spec( # noqa: PLR0913, PLR0917 + spec_str: str | None, + try_first_with: tuple[str, ...], + cache: PyInfoCache | None, + env: Mapping[str, str], + predicate: Callable[[PythonInfo], bool] | None, + seen: set[str], +) -> Iterator[PythonInfo]: + if spec_str is None: + spec = PythonSpec("", None, None, None, None, None, None) + wide = True + else: + spec = PythonSpec.from_string_spec(spec_str) + wide = False + for interpreter, impl_must_match in propose_interpreters( + spec, try_first_with, cache, env, all_implementations=wide + ): + if interpreter is None: + continue + if (anchor := interpreter.system_executable or interpreter.executable) is None: + continue + if (real_path := os.path.realpath(anchor)) in seen: + continue + if not interpreter.satisfies(spec, impl_must_match=impl_must_match): + continue + if predicate is not None and not predicate(interpreter): + continue + seen.add(real_path) + yield interpreter + + def _find_interpreter( key: str, try_first_with: Iterable[str], @@ -106,6 +175,8 @@ def propose_interpreters( try_first_with: Iterable[str], cache: PyInfoCache | None = None, env: Mapping[str, str] | None = None, + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: """ Yield ``(interpreter, impl_must_match)`` candidates for *spec*. @@ -114,6 +185,8 @@ def propose_interpreters( :param try_first_with: executable paths to probe before the standard search. :param cache: interpreter metadata cache; when ``None`` results are not cached. :param env: environment mapping for ``PATH`` lookup; defaults to :data:`os.environ`. + :param all_implementations: when ``True`` and *spec* does not constrain the implementation, also surface + non-CPython binaries on ``PATH`` and under UV's install directory. Used by enumeration APIs. """ env = os.environ if env is None else env tested_exes: set[str] = set() @@ -125,8 +198,8 @@ def propose_interpreters( yield from _propose_explicit(spec, try_first_with, cache, env, tested_exes) if spec.path is not None and spec.is_abs: # pragma: no cover # relative spec.path is never abs return - yield from _propose_from_path(spec, cache, env, tested_exes) - yield from _propose_from_uv(cache, env) + yield from _propose_from_path(spec, cache, env, tested_exes, all_implementations=all_implementations) + yield from _propose_from_uv(cache, env, all_implementations=all_implementations) def _propose_explicit( @@ -170,8 +243,10 @@ def _propose_from_path( cache: PyInfoCache | None, env: Mapping[str, str], tested_exes: set[str], + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: - find_candidates = path_exe_finder(spec) + find_candidates = path_exe_finder(spec, all_implementations=all_implementations) for pos, path in enumerate(get_paths(env)): _LOGGER.debug(LazyPathDump(pos, path, env)) for exe, impl_must_match in find_candidates(path): @@ -189,6 +264,8 @@ def _propose_from_path( def _propose_from_uv( cache: PyInfoCache | None, env: Mapping[str, str], + *, + all_implementations: bool = False, ) -> Generator[tuple[PythonInfo | None, bool], None, None]: if uv_python_dir := os.getenv("UV_PYTHON_INSTALL_DIR"): uv_python_path = Path(uv_python_dir).expanduser() @@ -197,10 +274,18 @@ def _propose_from_uv( else: uv_python_path = user_data_path("uv") / "python" - for exe_path in uv_python_path.glob("*/bin/python"): # pragma: no branch - interpreter = PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env) - if interpreter is not None: # pragma: no branch - yield interpreter, True + patterns: list[str] = ["*/bin/python", "*/python.exe"] + if all_implementations: + patterns.extend(("*/bin/pypy*", "*/bin/graalpy", "*/pypy*.exe", "*/bin/graalpy.exe")) + seen_uv_paths: set[str] = set() + for pattern in patterns: + for exe_path in uv_python_path.glob(pattern): + resolved = str(Path(exe_path).resolve()) + if resolved in seen_uv_paths: + continue + seen_uv_paths.add(resolved) + if interpreter := PathPythonInfo.from_exe(str(exe_path), cache, raise_on_error=False, env=env): + yield interpreter, True def get_paths(env: Mapping[str, str]) -> Generator[Path, None, None]: @@ -244,9 +329,11 @@ def __repr__(self) -> str: return content -def path_exe_finder(spec: PythonSpec) -> Callable[[Path], Generator[tuple[Path, bool], None, None]]: +def path_exe_finder( + spec: PythonSpec, *, all_implementations: bool = False +) -> Callable[[Path], Generator[tuple[Path, bool], None, None]]: """Given a spec, return a function that can be called on a path to find all matching files in it.""" - pat = spec.generate_re(windows=sys.platform == "win32") + pat = spec.generate_re(windows=sys.platform == "win32", all_implementations=all_implementations) direct = spec.str_spec if sys.platform == "win32": # pragma: win32 cover direct = f"{direct}.exe" @@ -331,5 +418,6 @@ class PathPythonInfo(PythonInfo): "PathPythonInfo", "get_interpreter", "get_paths", + "iter_interpreters", "propose_interpreters", ] diff --git a/src/python_discovery/_py_spec.py b/src/python_discovery/_py_spec.py index ebdea59..e3cf590 100644 --- a/src/python_discovery/_py_spec.py +++ b/src/python_discovery/_py_spec.py @@ -35,6 +35,8 @@ _MAX_VERSION_PARTS: Final[int] = 3 _SINGLE_DIGIT_MAX: Final[int] = 9 +KNOWN_IMPLEMENTATIONS: Final[tuple[str, ...]] = ("python", "cpython", "pypy", "graalpy") + SpecifierSet = SimpleSpecifierSet Version = SimpleVersion InvalidSpecifier = ValueError @@ -111,7 +113,9 @@ class PythonSpec: :param path: filesystem path to a specific interpreter, or ``None``. :param free_threaded: whether a free-threaded build is required, or ``None`` for any. :param machine: required ISA (e.g. ``"arm64"``), or ``None`` for any. - :param version_specifier: PEP 440 version constraints, or ``None``. + :param version_specifier: + `version specifier `_ + constraints, or ``None``. """ def __init__( # noqa: PLR0913, PLR0917 @@ -145,7 +149,7 @@ def from_string_spec(cls, string_spec: str) -> PythonSpec: Parse a string specification into a :class:`PythonSpec`. :param string_spec: an interpreter spec — an absolute path, a version string, an implementation prefix, - or a PEP 440 specifier. + or a `version specifier `_. """ if pathlib.Path(string_spec).is_absolute(): return cls(string_spec, None, None, None, None, None, string_spec) @@ -155,16 +159,23 @@ def from_string_spec(cls, string_spec: str) -> PythonSpec: return result return cls(string_spec, None, None, None, None, None, string_spec) - def generate_re(self, *, windows: bool) -> re.Pattern: + def generate_re(self, *, windows: bool, all_implementations: bool = False) -> re.Pattern: """ Generate a regular expression for matching interpreter filenames. :param windows: if ``True``, require a ``.exe`` suffix. + :param all_implementations: when ``True`` and the spec does not constrain the implementation, match every + filename in :data:`KNOWN_IMPLEMENTATIONS` instead of only ``python``. Used by enumeration APIs. """ version = r"{}(\.{}(\.{})?)?".format( *(r"\d+" if v is None else v for v in (self.major, self.minor, self.micro)), ) - impl = "python" if self.implementation is None else f"python|{re.escape(self.implementation)}" + if self.implementation is not None: + impl = f"python|{re.escape(self.implementation)}" + elif all_implementations: + impl = "|".join(re.escape(i) for i in KNOWN_IMPLEMENTATIONS) + else: + impl = "python" mod = "t?" if self.free_threaded else "" suffix = r"\.exe" if windows else "" version_conditional = "?" if windows or self.major is None else "" @@ -254,6 +265,7 @@ def __repr__(self) -> str: __all__ = [ + "KNOWN_IMPLEMENTATIONS", "InvalidSpecifier", "InvalidVersion", "PythonSpec", diff --git a/tests/test_iter_interpreters.py b/tests/test_iter_interpreters.py new file mode 100644 index 0000000..5f50a8b --- /dev/null +++ b/tests/test_iter_interpreters.py @@ -0,0 +1,218 @@ +from __future__ import annotations + +import contextlib +import os +import sys +from pathlib import Path +from typing import TYPE_CHECKING +from uuid import uuid4 + +import pytest + +from python_discovery import DiskCache, PythonInfo, iter_interpreters +from python_discovery._discovery import IS_WIN + +if TYPE_CHECKING: + from collections.abc import Callable, Iterator + + from pytest_mock import MockerFixture + + +@pytest.fixture +def uv_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Empty uv install root with environment variables pointing at it.""" + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + root = tmp_path / "uv-python" + root.mkdir() + monkeypatch.setenv("UV_PYTHON_INSTALL_DIR", str(root)) + return root + + +def test_iter_interpreters_returns_iterator(session_cache: DiskCache) -> None: + result: Iterator[PythonInfo] = iter_interpreters(sys.executable, cache=session_cache) + assert iter(result) is result + + +def test_iter_interpreters_with_abs_path_yields_one(session_cache: DiskCache) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache)) + assert len(results) == 1 + assert results[0].executable == sys.executable + + +def test_iter_interpreters_sequence_dedups_across_keys(session_cache: DiskCache) -> None: + results = list(iter_interpreters([sys.executable, sys.executable], cache=session_cache)) + assert len(results) == 1 + + +@pytest.mark.parametrize( + ("predicate", "expected_count"), + [ + pytest.param(None, 1, id="none-is-noop"), + pytest.param(lambda _: True, 1, id="accepts-all"), + pytest.param(lambda _: False, 0, id="rejects-all"), + ], +) +def test_iter_interpreters_predicate( + session_cache: DiskCache, + predicate: Callable[[PythonInfo], bool] | None, + expected_count: int, +) -> None: + results = list(iter_interpreters(sys.executable, cache=session_cache, predicate=predicate)) + assert len(results) == expected_count + + +@pytest.mark.parametrize( + "key_factory", + [ + pytest.param(lambda: uuid4().hex, id="filename-cannot-match"), + pytest.param(lambda: ">=999", id="version-cannot-match"), + ], +) +def test_iter_interpreters_filters_to_empty( + session_cache: DiskCache, + key_factory: Callable[[], str], +) -> None: + assert list(iter_interpreters(key_factory(), cache=session_cache)) == [] + + +@pytest.mark.usefixtures("uv_dir") +def test_iter_interpreters_no_key_includes_running_interpreter(session_cache: DiskCache) -> None: + real_self = os.path.realpath(sys.executable) + results = list(iter_interpreters(cache=session_cache)) + assert any(os.path.realpath(info.executable) == real_self for info in results) + + +@pytest.mark.usefixtures("uv_dir") +def test_iter_interpreters_dedups_symlinks( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + session_cache: DiskCache, +) -> None: + suffix = ".exe" if IS_WIN else "" + for name in ("python3", "python3.99"): + Path(str(tmp_path / f"{name}{suffix}")).symlink_to(sys.executable) + pyvenv_cfg = Path(sys.executable).parents[1] / "pyvenv.cfg" + with contextlib.suppress(FileNotFoundError): + (tmp_path / pyvenv_cfg.name).write_bytes(pyvenv_cfg.read_bytes()) + monkeypatch.setenv("PATH", os.pathsep.join([str(tmp_path), os.environ.get("PATH", "")])) + + real = os.path.realpath(sys.executable) + matches = [info for info in iter_interpreters(cache=session_cache) if os.path.realpath(info.executable) == real] + assert len(matches) == 1 + + +@pytest.mark.usefixtures("uv_dir") +def test_iter_interpreters_try_first_with_yields_first( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + session_cache: DiskCache, +) -> None: + monkeypatch.setenv("PATH", str(tmp_path)) + results = list(iter_interpreters(try_first_with=[sys.executable], cache=session_cache)) + assert results + assert os.path.realpath(results[0].executable) == os.path.realpath(sys.executable) + + +def test_iter_interpreters_skips_none_from_propose( + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + real_info = PythonInfo.current(session_cache) + mocker.patch( + "python_discovery._discovery.propose_interpreters", + return_value=iter([(None, True), (real_info, True)]), + ) + results = list(iter_interpreters(cache=session_cache)) + assert results == [real_info] + + +def test_iter_interpreters_skips_when_no_executable( + session_cache: DiskCache, + mocker: MockerFixture, +) -> None: + bogus = mocker.MagicMock(spec=PythonInfo) + bogus.system_executable = None + bogus.executable = None + mocker.patch( + "python_discovery._discovery.propose_interpreters", + return_value=iter([(bogus, True)]), + ) + assert list(iter_interpreters(cache=session_cache)) == [] + + +def test_iter_interpreters_uv_yields_when_interrogation_succeeds( + uv_dir: Path, + session_cache: DiskCache, + monkeypatch: pytest.MonkeyPatch, + mocker: MockerFixture, +) -> None: + monkeypatch.setenv("PATH", "") + bin_path = uv_dir / "cpython-3.99-fake/bin" + bin_path.mkdir(parents=True) + (bin_path / "python").touch() + + fake_info = mocker.MagicMock(spec=PythonInfo) + fake_info.system_executable = str(uv_dir / "fake-unique" / "python") + fake_info.executable = fake_info.system_executable + fake_info.satisfies.return_value = True + mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=fake_info) + + assert fake_info in list(iter_interpreters(cache=session_cache)) + + +@pytest.mark.parametrize( + ("real_files", "symlinks", "expected_real"), + [ + pytest.param( + ("pypy-3.10-linux-x86_64-gnu/bin/pypy3.10",), + ( + ("pypy-3.10-linux-x86_64-gnu/bin/python", "pypy-3.10-linux-x86_64-gnu/bin/pypy3.10"), + ("pypy-3.10-linux-x86_64-gnu/bin/pypy", "pypy-3.10-linux-x86_64-gnu/bin/pypy3.10"), + ), + ("pypy-3.10-linux-x86_64-gnu/bin/pypy3.10",), + id="pypy-unix-with-python-symlink", + ), + pytest.param( + ("graalpy-24.1.1-linux-x86_64-gnu/bin/graalpy",), + (), + ("graalpy-24.1.1-linux-x86_64-gnu/bin/graalpy",), + id="graalpy-unix", + ), + pytest.param( + ( + "cpython-3.14.4-windows-x86_64-none/python.exe", + "pypy-3.11.15-windows-x86_64-none/pypy3.11.exe", + "graalpy-3.11-windows-x86_64-none/bin/graalpy.exe", + ), + (), + ( + "cpython-3.14.4-windows-x86_64-none/python.exe", + "pypy-3.11.15-windows-x86_64-none/pypy3.11.exe", + "graalpy-3.11-windows-x86_64-none/bin/graalpy.exe", + ), + id="windows-all-impls", + ), + ], +) +def test_iter_interpreters_uv_layout( + uv_dir: Path, + session_cache: DiskCache, + monkeypatch: pytest.MonkeyPatch, + mocker: MockerFixture, + real_files: tuple[str, ...], + symlinks: tuple[tuple[str, str], ...], + expected_real: tuple[str, ...], +) -> None: + monkeypatch.setenv("PATH", "") + for rel in real_files: + path = uv_dir / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.touch() + for link_rel, target_rel in symlinks: + Path(str(uv_dir / link_rel)).symlink_to(uv_dir / target_rel) + + mock_from_exe = mocker.patch("python_discovery._discovery.PathPythonInfo.from_exe", return_value=None) + list(iter_interpreters(cache=session_cache)) + + interrogated_real = {os.path.realpath(call.args[0]) for call in mock_from_exe.call_args_list} + assert interrogated_real == {os.path.realpath(uv_dir / r) for r in expected_real}