Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ The `get_interpreter()` function accepts various specification formats:
- Absolute path: `/usr/bin/python3.12`
- Version: `3.12`
- Implementation prefix: `cpython3.12`
- PEP 440 specifier: `>=3.10`, `>=3.11,<3.13`
- [Version specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/): `>=3.10`,
`>=3.11,<3.13`

## Documentation

Expand Down
3 changes: 3 additions & 0 deletions docs/changelog/65.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
discover uv-managed Pythons on Windows. Previously the glob assumed Unix layout (``<root>/<key>/bin/python``) and
silently found nothing on Windows, where uv places ``python.exe`` directly under the install root - by
:user:`gaborbernat`.
3 changes: 3 additions & 0 deletions docs/changelog/65.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
add :func:`~python_discovery.iter_interpreters` for enumerating every discovered interpreter, with PATH and
UV-install support for non-CPython implementations listed in :data:`~python_discovery.KNOWN_IMPLEMENTATIONS`
- by :user:`gaborbernat`.
111 changes: 107 additions & 4 deletions docs/explanation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,106 @@ detects these shims and resolves them to the actual binary.
`mise <https://mise.jdx.dev/>`_ and `asdf <https://asdf-vm.com/>`_ work similarly, using the
``MISE_DATA_DIR`` and ``ASDF_DATA_DIR`` environment variables to locate their installations.

How uv-managed Pythons are discovered
---------------------------------------

`uv <https://docs.astral.sh/uv/>`_ installs Python interpreters under a single root directory (configurable via
``UV_PYTHON_INSTALL_DIR``, otherwise defaulting under ``XDG_DATA_HOME`` or the platform user-data path). Each
install lives in its own subdirectory, but the actual binary location varies by OS and implementation:

.. list-table::
:header-rows: 1
:widths: 25 35 40

* - Implementation
- Unix layout
- Windows layout
* - CPython
- ``<root>/<key>/bin/python``
- ``<root>/<key>/python.exe``
* - PyPy
- ``<root>/<key>/bin/pypy*``
- ``<root>/<key>/pypy*.exe``
* - GraalPy
- ``<root>/<key>/bin/graalpy``
- ``<root>/<key>/bin/graalpy.exe``

.. mermaid::

flowchart LR
Call(["iter_interpreters(key)"]) --> Mode{"key is None?"}
Mode -->|"narrow"| N1["*/bin/python"]
Mode -->|"narrow"| N2["*/python.exe"]
Mode -->|"wide"| W1["*/bin/pypy*"]
Mode -->|"wide"| W2["*/bin/graalpy"]
Mode -->|"wide"| W3["*/pypy*.exe"]
Mode -->|"wide"| W4["*/bin/graalpy.exe"]

N1 --> Dedup[/"realpath dedup"/]
N2 --> Dedup
W1 --> Dedup
W2 --> Dedup
W3 --> Dedup
W4 --> Dedup

Dedup --> Interrogate(["subprocess interrogation"])

style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff
style Mode fill:#d9904a,stroke:#8f5f2a,color:#fff
style N1 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
style N2 fill:#3a7fc2,stroke:#1f4d7a,color:#fff
style W1 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
style W2 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
style W3 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
style W4 fill:#9f4ad9,stroke:#5f2a8f,color:#fff
style Dedup fill:#c2873a,stroke:#7a4c1f,color:#fff
style Interrogate fill:#4a9f4a,stroke:#2a6f2a,color:#fff

GraalPy keeps its ``bin/`` segment on Windows (an upstream choice in uv); PyPy and CPython do not. python-discovery
globs all of these patterns regardless of the host OS, because globs that do not match anything are essentially
free, and the cross-platform list is short. Symlinked aliases inside an install (``bin/python``,
``bin/python3``, ``bin/python3.14`` all pointing at the same real file) are deduplicated by resolved path before
the subprocess interrogation, so each install is interrogated once.

Selecting one interpreter vs. enumerating all of them
-------------------------------------------------------

:func:`~python_discovery.get_interpreter` and :func:`~python_discovery.iter_interpreters` walk the same candidate
sources, but they answer different questions and behave differently in three ways.

.. mermaid::

flowchart LR
Sources["candidate sources<br>(try_first_with → current →<br>PEP 514 → PATH → uv)"]
Sources --> Get["get_interpreter()<br>first match wins, returns one"]
Sources --> Iter["iter_interpreters()<br>yields every match"]

style Get fill:#4a9f4a,stroke:#2a6f2a,color:#fff
style Iter fill:#4a90d9,stroke:#2a5f8f,color:#fff

**Implementation coverage on PATH.** :func:`~python_discovery.get_interpreter` matches only ``python*`` filenames on
PATH unless the spec names another implementation explicitly (``pypy3.12``, ``graalpy3.11``). This keeps backwards
compatibility with tools that have always read "no implementation in the spec" as "give me CPython."
:func:`~python_discovery.iter_interpreters` with no spec broadens the search to every name in
:data:`~python_discovery.KNOWN_IMPLEMENTATIONS` -- otherwise an "all interpreters" call would silently miss every
PyPy and GraalPy on the system. When you pass a spec to :func:`~python_discovery.iter_interpreters`, it falls back
to the same narrow regex as :func:`~python_discovery.get_interpreter`, so behavior is consistent across the two
APIs whenever a spec is given.

**Deduplication.** :func:`~python_discovery.get_interpreter` deduplicates per call so it does not interrogate the
same binary twice while searching, and stops as soon as a match is found. :func:`~python_discovery.iter_interpreters`
deduplicates by the resolved real path of each candidate's ``system_executable`` (falling back to ``executable``).
That means symlinked aliases like ``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv whose ``python``
symlinks to its base interpreter, collapse to a single yield. The semantic is "one entry per distinct install,"
which is what callers building choosers or version-range pickers usually want.

**Iteration order.** Yields come back in *priority order*: ``try_first_with`` first, then the running interpreter,
then :pep:`514` entries on Windows, then PATH left-to-right, then UV-managed installs. This matches what
:func:`~python_discovery.get_interpreter` would have returned at each step. If your ordering differs (newest
version first, smallest install root, etc.), wrap the call in :func:`sorted` -- the API deliberately does not
include a ``sort_by`` parameter because keeping discovery order preserves the priority signal for callers who
want it.

How caching works
-------------------

Expand Down Expand Up @@ -165,9 +265,12 @@ A spec string follows the pattern ``[impl][version][t][-arch][-machine]``. Every
* - ``/usr/bin/python3``
- Absolute path, used directly (no search)
* - ``>=3.11,<3.13``
- :pep:`440` version specifier (any Python in range)
- `Version specifier <https://packaging.python.org/en/latest/specifications/version-specifiers/>`_
(any Python in range)
* - ``cpython>=3.11``
- :pep:`440` specifier restricted to CPython
- `Version specifier <https://packaging.python.org/en/latest/specifications/version-specifiers/>`_
restricted to CPython

:pep:`440` specifiers (``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple
specifiers can be comma-separated, for example ``>=3.11,<3.13``.
`Version specifiers <https://packaging.python.org/en/latest/specifications/version-specifiers/>`_
(``>=``, ``<=``, ``~=``, ``!=``, ``==``, ``===``) are supported. Multiple specifiers can be comma-separated,
for example ``>=3.11,<3.13``.
54 changes: 54 additions & 0 deletions docs/how-to/standalone-usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,60 @@ the ``PY_DISCOVERY_TIMEOUT`` environment variable.
The timeout value should be a number in seconds. Each interpreter candidate is given this much time
to respond. If a timeout occurs, the candidate is skipped and the search continues with the next one.

List every interpreter on the system
--------------------------------------

Use :func:`~python_discovery.iter_interpreters` to enumerate every Python python-discovery can find. With no spec
it yields all known implementations (CPython, PyPy, GraalPy -- see
:data:`~python_discovery.KNOWN_IMPLEMENTATIONS`). Pass a spec to filter, exactly like
:func:`~python_discovery.get_interpreter`.

.. code-block:: python

from pathlib import Path

from python_discovery import DiskCache, iter_interpreters

cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser())

# Every interpreter, no filter
for info in iter_interpreters(cache=cache):
print(info.executable, info.version_str, info.implementation)

# Every CPython 3.10 or newer, newest first
newest_first = sorted(
iter_interpreters("cpython>=3.10", cache=cache),
key=lambda info: info.version_info,
reverse=True,
)

Enumeration interrogates every candidate as a subprocess on a cold cache. Always pass a
:class:`~python_discovery.DiskCache` if you call this more than once.

Pick an interpreter from a range, preferring newer
----------------------------------------------------

A common need: search a version range and prefer newer interpreters when more than one matches. Sort the result of
:func:`~python_discovery.iter_interpreters` and take the first.

.. code-block:: python

from pathlib import Path

from python_discovery import DiskCache, iter_interpreters

cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser())

matches = sorted(
iter_interpreters("cpython>=3.10,<3.15", cache=cache),
key=lambda info: info.version_info,
reverse=True,
)
info = matches[0] if matches else None

Use :func:`get_interpreter` instead when you only need the first PATH-priority hit; use the sort-and-take pattern
when *your* ordering differs from PATH order (newest version, smallest install size, preferred install root, etc.).

Read interpreter metadata
---------------------------

Expand Down
46 changes: 45 additions & 1 deletion docs/tutorial/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,49 @@ You can pass multiple specs as a list -- the library tries each one in order and

result = get_interpreter(["python3.12", "python3.11"], cache=cache)

Listing every interpreter
---------------------------

When you need *every* interpreter rather than just the first match -- for example, to show the user a chooser, or
to apply your own ranking -- use :func:`~python_discovery.iter_interpreters`. Pass no arguments to enumerate every
implementation python-discovery knows about, or pass a spec to filter.

.. mermaid::

flowchart TD
Call["iter_interpreters(spec, cache)"] --> Yield["yields PythonInfo"]
Yield --> A["1. try_first_with paths"]
Yield --> B["2. running interpreter"]
Yield --> C["3. PATH (left to right)"]
Yield --> D["4. uv-managed installs"]

style Call fill:#4a90d9,stroke:#2a5f8f,color:#fff
style Yield fill:#4a9f4a,stroke:#2a6f2a,color:#fff

.. code-block:: python

from pathlib import Path

from python_discovery import DiskCache, iter_interpreters

cache = DiskCache(root=Path("~/.cache/python-discovery").expanduser())
for info in iter_interpreters(cache=cache):
print(info.executable, info.version_str, info.implementation)

The result is an iterator, so :func:`list`, :func:`sorted`, generator expressions and early ``break`` all work as you
would expect. Symlinked aliases (``/bin/python3`` and ``/usr/bin/python3``, or a virtualenv and the base it points at)
collapse to a single entry, so you do not see the same install twice.

To prefer newer interpreters in a range, sort the result by ``version_info`` after filtering:

.. code-block:: python

newest_first = sorted(
iter_interpreters(">=3.10,<3.15", cache=cache),
key=lambda info: info.version_info,
reverse=True,
)

Writing specs
-------------

Expand Down Expand Up @@ -131,7 +174,8 @@ Common examples:
* - ``/usr/bin/python3``
- An absolute path, used directly without searching
* - ``>=3.11,<3.13``
- Any Python in the 3.11--3.12 range (:pep:`440` syntax)
- Any Python in the 3.11--3.12 range
(`version specifier <https://packaging.python.org/en/latest/specifications/version-specifiers/>`_ syntax)

See the :doc:`full spec reference </explanation>` for all options.

Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ lint.per-file-ignores."tests/**/*.py" = [
"PLC0415", # imports inside test functions (conditional on mocking)
"PLC2701", # private imports needed to test internal APIs
"PLR0913", # too many arguments (pytest fixtures)
"PLR0917", # too many positional arguments (pytest fixtures)
"PLR2004", # Magic value used in comparison
"S101", # asserts allowed in tests
"S404", # subprocess import
Expand Down
6 changes: 4 additions & 2 deletions src/python_discovery/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@
from importlib.metadata import version

from ._cache import ContentStore, DiskCache, PyInfoCache
from ._discovery import get_interpreter
from ._discovery import get_interpreter, iter_interpreters
from ._py_info import KNOWN_ARCHITECTURES, PythonInfo, normalize_isa
from ._py_spec import PythonSpec
from ._py_spec import KNOWN_IMPLEMENTATIONS, PythonSpec
from ._specifier import SimpleSpecifier, SimpleSpecifierSet, SimpleVersion

__version__ = version("python-discovery")

__all__ = [
"KNOWN_ARCHITECTURES",
"KNOWN_IMPLEMENTATIONS",
"ContentStore",
"DiskCache",
"PyInfoCache",
Expand All @@ -24,5 +25,6 @@
"SimpleVersion",
"__version__",
"get_interpreter",
"iter_interpreters",
"normalize_isa",
]
Loading