Skip to content

fix(workflows): auto-detect project integration instead of hardcoding copilot default#2408

Open
markuswondrak wants to merge 24 commits intogithub:mainfrom
markuswondrak:fix/workflow-integration-auto-detect
Open

fix(workflows): auto-detect project integration instead of hardcoding copilot default#2408
markuswondrak wants to merge 24 commits intogithub:mainfrom
markuswondrak:fix/workflow-integration-auto-detect

Conversation

@markuswondrak
Copy link
Copy Markdown

@markuswondrak markuswondrak commented Apr 29, 2026

Description

Fixes #2406

The built-in speckit workflow used "copilot" as the default integration input. In projects initialized with a different integration such as opencode, workflows could dispatch to the wrong agent and fail with "No such agent: speckit.specify" when the copilot CLI happened to be installed locally.

Changes

  1. workflows/speckit/workflow.yml

    • Changed the integration input default from "copilot" to "auto"
    • Updated the prompt examples to include opencode and the explicit auto option
    • Removed the stale static requires.integrations.any allowlist
  2. src/specify_cli/workflows/engine.py

    • Added _resolve_default() and _load_project_integration() to read .specify/integration.json
    • Preserved the "copilot" fallback for missing, unreadable, invalid, or empty config
    • Added post-processing in _resolve_inputs() so both defaulted and explicit --input integration=auto values resolve through the same path
  3. tests/test_workflows.py

    • Added 7 regression tests covering default auto-detection, explicit integration=auto, explicit override precedence, missing file fallback, invalid JSON fallback, OSError fallback, and whitespace-only values

How it works

  • The workflow YAML now defaults integration to "auto"
  • When the resolved integration value is "auto", the engine reads .specify/integration.json and uses the stored integration key
  • If the file is missing, unreadable, invalid, or does not contain a usable integration value, the engine falls back to "copilot" for backwards compatibility

Testing

  • uvx ruff check src/ tests/test_workflows.py
  • uv run pytest tests/test_workflows.py
  • uv run pytest tests/test_workflows.py -k TestIntegrationAutoDetect -v

AI Disclosure

  • I did use AI assistance (describe below)

This PR was authored with GitHub Copilot CLI assistance. All changes were reviewed and verified by the contributor.

… copilot (github#2406)

The workflow engine hardcoded 'copilot' as the default integration input,
ignoring the project's configured integration in .specify/integration.json.
This caused workflows to fail when a project was initialized with a different
integration (e.g. opencode) but the copilot CLI happened to be installed.

Changes:
- Change workflow.yml integration default from 'copilot' to 'auto'
- Add _resolve_integration_auto() to WorkflowEngine that reads
  .specify/integration.json and falls back to 'copilot' if absent
- Post-process resolved inputs to replace 'auto' with detected value
- Add 4 regression tests for auto-detection scenarios

Fixes github#2406

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 29, 2026 13:30
@markuswondrak markuswondrak requested a review from mnriem as a code owner April 29, 2026 13:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the built-in speckit workflow and workflow engine to avoid hardcoding "copilot" as the default integration by introducing an "auto" integration mode that resolves from project configuration.

Changes:

  • Set workflows/speckit/workflow.yml integration input default to "auto" and updated the prompt to document it.
  • Added WorkflowEngine._resolve_integration_auto() and post-processing in _resolve_inputs() to read .specify/integration.json, with fallback to "copilot".
  • Added regression tests for auto-detection, fallback behavior, explicit override precedence, and empty JSON handling; added an Unreleased changelog entry.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
workflows/speckit/workflow.yml Switches workflow input default integration from "copilot" to "auto" so runs honor project configuration.
src/specify_cli/workflows/engine.py Implements "auto" integration resolution by reading .specify/integration.json with a backward-compatible fallback.
tests/test_workflows.py Adds regression tests to ensure "auto" resolves correctly and preserves override behavior.
CHANGELOG.md Notes the fix in the Unreleased section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread tests/test_workflows.py Outdated
Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking into this. Please address Copilot feedback. If not applicable, please explain why

Copilot AI review requested due to automatic review settings April 29, 2026 14:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@markuswondrak markuswondrak requested a review from mnriem April 29, 2026 14:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 4

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread workflows/speckit/workflow.yml Outdated
Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

- Add tests for JSONDecodeError and OSError fallback paths in _resolve_integration_auto()
- Centralize INTEGRATION_JSON constant in workflows/constants.py (zero-dependency module)
- Change changelog reference to (Fixes github#2406) for auto-close semantics
- Remove static requires.integrations.any from speckit workflow (incomplete with 'auto')
- Add opencode to integration prompt examples
@markuswondrak
Copy link
Copy Markdown
Author

Thanks for the thorough review! I've addressed all four findings:

  1. Missing test coverage — Added two new tests for JSONDecodeError and OSError fallback paths in _resolve_integration_auto().

  2. Duplicated constant — Centralized INTEGRATION_JSON into src/specify_cli/workflows/constants.py (zero-dependency module) and imported from both engine.py and __init__.py.

  3. Changelog reference — Changed (#2406) to (Fixes #2406) for auto-close semantics.

  4. Workflow metadata — Removed requires.integrations.any from workflow.yml. Since "auto" resolves the integration dynamically from the project's integration.json, a static allowlist is inherently incomplete — every new integration added to the registry would require updating this workflow file, creating clutter and a maintenance burden. The requires field is also not enforced at runtime currently (engine.py line 50-52: "declared but not yet enforced at runtime; enforcement is a planned enhancement"). If/when enforcement is added, it should validate against the runtime INTEGRATION_REGISTRY rather than a hardcoded list.

    Is removing the integrations key entirely acceptable, or would you prefer to keep it as documentation with the full list of all 26 integrations?

@markuswondrak markuswondrak requested a review from mnriem April 29, 2026 17:19
Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve conflicts

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/6 changed files
  • Comments generated: 3

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/__init__.py Outdated
Comment thread tests/test_workflows.py Outdated
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented Apr 29, 2026

Please revert changes to CHANGELOG.md as it is now auto-generated (still need to update docs to no longer require it) and resolve the other conflict

Markus and others added 2 commits April 30, 2026 07:08
- CHANGELOG.md: take released [0.8.3] block from origin, discard
  auto-generated [Unreleased] section that shouldn't have been committed
- __init__.py: add devin_skill_mode from origin, preserve multi-line
  native_skill_mode format from HEAD

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- engine.py: strip whitespace from integration value in
  _resolve_integration_auto(); whitespace-only strings like '   '
  now fall back to 'copilot' instead of being returned as-is
- __init__.py: define INTEGRATION_JSON locally instead of importing
  from workflows.constants; avoids executing the workflows package
  __init__ (which registers all step types) during CLI startup for
  commands that don't use workflows
- tests: add test for whitespace-only integration value fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/constants.py Outdated
Move INTEGRATION_JSON from specify_cli.workflows.constants to
specify_cli.constants to avoid importing the workflows package
when the main CLI module loads.

Update imports in __init__.py and workflows/engine.py accordingly,
and remove the now-empty workflows/constants.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Markus and others added 3 commits May 3, 2026 09:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread workflows/speckit/workflow.yml Outdated
Comment thread src/specify_cli/paths.py Outdated
Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/workflows/engine.py
markuswondrak pushed a commit to markuswondrak/spec-kit that referenced this pull request May 3, 2026
- engine.py: Import INTEGRATION_JSON from integration_state (single source of truth)
- engine.py: _load_project_integration() uses default_integration_key() to properly
  resolve default_integration -> integration key resolution
- engine.py: _resolve_inputs() resolves 'auto' sentinel before enum validation so
  workflows with enum constraints on integration input work correctly
- engine.py: StepContext creation resolves 'auto' in default_integration so context
  always carries a real integration key
- paths.py: Remove duplicate INTEGRATION_JSON constant
- workflow.yml: Bump speckit_version to >=0.8.4 (older CLIs don't support 'auto')
- tests: Add 5 regression tests for the fixes

Fixes review comments from copilot-pull-request-reviewer[bot] on PR github#2408
@markuswondrak markuswondrak requested a review from Copilot May 3, 2026 13:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread workflows/speckit/workflow.yml Outdated
Comment thread tests/test_workflows.py Outdated
Comment thread src/specify_cli/paths.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 3, 2026 14:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/paths.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread workflows/speckit/workflow.yml
…etect

Issue 1 (engine.py:726): Explicit --input integration=auto was rejected by
enum validation before the sentinel could be resolved. Move the auto-resolution
to before _coerce_input() so enum-constrained inputs also accept 'auto'.

Issue 2 (engine.py:737): workflow.integration: auto in the YAML was stored
raw as 'auto' in WorkflowDefinition.default_integration and leaked into
StepContext, causing step dispatch to look for an 'auto' CLI. Add
_resolve_workflow_integration() helper and use it at both StepContext
construction sites.

Issue 3 (engine.py:779): _load_project_integration() only queried the legacy
'integration' field. Newer state files may carry 'default_integration' as the
canonical key. Replace the custom inner reader for integration.json with a
call to the shared default_integration_key() from integration_state, which
handles both fields and their precedence correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/specify_cli/workflows/engine.py
Comment thread src/specify_cli/workflows/engine.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

src/specify_cli/workflows/engine.py:731

  • The default branch still bypasses _coerce_input(). For workflows that declare integration.default: auto together with an enum allowlist, a project initialized with an unsupported integration will skip enum validation entirely and dispatch to that unsupported CLI. The resolved default needs to be validated after expanding auto, not just user-provided values.
            elif "default" in input_def:
                resolved[name] = self._resolve_default(name, input_def["default"])

src/specify_cli/workflows/engine.py:800

  • This is not actually using the shared normalized integration-state reader. The rest of the CLI goes through _read_integration_json() (src/specify_cli/__init__.py:1921-1950), which normalizes legacy state before resolving the default integration. Parsing raw JSON here means states that rely on normalization (for example, deriving the default from installed_integrations) will still fall through to init-options.json/copilot in workflows.
        # Primary source: .specify/integration.json — use the shared normalized
        # reader so both "integration" and "default_integration" fields are
        # handled and future schema versions are respected.
        json_path = self.project_root / _INTEGRATION_JSON
        if json_path.is_file():
            try:
                data = json.loads(json_path.read_text(encoding="utf-8"))
            except (OSError, UnicodeDecodeError, json.JSONDecodeError):
                data = None
            if isinstance(data, dict):
                key = _default_integration_key(data)
                if key and key != "auto":
                    return key
  • Files reviewed: 5/5 changed files
  • Comments generated: 1

Comment on lines +482 to +484
default_integration=self._resolve_workflow_integration(
definition.default_integration
),
Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Workflow engine hardcodes "copilot" default, ignoring project's initialized integration (e.g., opencode)

4 participants