Skip to content

fix(workflow): support integration: auto to follow project's initialized AI#2421

Open
Quratulain-bilal wants to merge 2 commits intogithub:mainfrom
Quratulain-bilal:fix/workflow-integration-auto-default
Open

fix(workflow): support integration: auto to follow project's initialized AI#2421
Quratulain-bilal wants to merge 2 commits intogithub:mainfrom
Quratulain-bilal:fix/workflow-integration-auto-default

Conversation

@Quratulain-bilal
Copy link
Copy Markdown
Contributor

@Quratulain-bilal Quratulain-bilal commented Apr 30, 2026

Closes #2406

Problem

The speckit workflow's integration input had default: "copilot" hardcoded in workflows/speckit/workflow.yml. This meant projects initialized with claude or gemini would silently fall back to copilot when running the workflow without an explicit --input integration=... flag, breaking the principle of least surprise.

Fix

  • workflows/speckit/workflow.yml: change default: "copilot"default: "auto", update the prompt to describe the new behavior, and broaden requires.integrations.any to reflect that the workflow's four core commands (specify/plan/tasks/implement) are provided by every integration.
  • src/specify_cli/_paths.py (new): single-source-of-truth for INTEGRATION_JSON and a shared low-level try_read_integration_json helper used by both the CLI and the workflow engine.
  • src/specify_cli/workflows/engine.py:
    • Add _resolve_default() so when an input default is the sentinel string "auto" and the input name is "integration", the engine reads .specify/integration.json (via the shared helper) and uses the project's initialized integration. Falls back to the literal "auto" when the file is missing or malformed.
    • Run resolved defaults through _coerce_input() so workflow defaults are validated against declared type/enum constraints, just like user-provided inputs.
    • Extend validate_workflow() to validate input defaults at authoring time so enum/type mismatches surface during install/validation rather than at execution. The integration: auto sentinel is exempt because it is runtime-resolved against project state.
  • src/specify_cli/__init__.py: re-export INTEGRATION_JSON from _paths for backward compatibility.
  • tests/test_workflows.py: 8 new regression tests covering the auto-resolve happy path, the missing-file fallback, the explicit-input override, the malformed JSON path, the non-UTF8 path, default coercion, default enum validation, and the validate_workflow exemption for auto.

The change is opt-in via the YAML default, so existing workflows that hardcode default: "copilot" (or any other literal) keep working unchanged.

Test plan

  • All 135 workflow tests pass locally
  • Full test suite passes locally (1732 passed, 125 skipped)
  • speckit workflow respects .specify/integration.json when no --input is given
  • Explicit --input integration=copilot still overrides the project default
  • Missing, malformed, or non-UTF8 .specify/integration.json does not crash the engine
  • Authoring-time validate_workflow() rejects defaults that violate declared enum/type

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts the speckit workflow so its integration input defaults to the project’s initialized AI integration (via .specify/integration.json) instead of being hardcoded to Copilot, aligning workflow execution with project initialization state.

Changes:

  • Updated workflows/speckit/workflow.yml to use default: "auto" for inputs.integration and clarified the prompt text.
  • Enhanced WorkflowEngine input resolution to interpret the "auto" default sentinel for integration by reading .specify/integration.json.
  • Added regression tests covering auto-resolution, explicit override behavior, missing file fallback, and malformed JSON fallback.
Show a summary per file
File Description
workflows/speckit/workflow.yml Switches the workflow’s integration default to auto and documents the behavior in the input prompt.
src/specify_cli/workflows/engine.py Implements "auto" default resolution for the integration input by reading .specify/integration.json.
tests/test_workflows.py Adds tests validating the new default resolution and fallback/override behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 3

Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread workflows/speckit/workflow.yml
Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

Quratulain-bilal added a commit to Quratulain-bilal/spec-kit that referenced this pull request May 1, 2026
- Centralize the integration.json path as a module-level INTEGRATION_JSON
  constant in workflows/engine.py (mirrors specify_cli.INTEGRATION_JSON;
  cannot be imported directly without a circular dependency).
- Catch UnicodeDecodeError alongside JSONDecodeError/OSError so a
  non-UTF8 integration.json falls back to the literal default instead
  of crashing the workflow engine. Adds a regression test.
- Drop the narrow requires.integrations.any allowlist in the speckit
  workflow YAML; the four core commands (specify/plan/tasks/implement)
  are provided by every integration, so the previous list was always
  inaccurate (e.g. opencode, codex, etc. were excluded).
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

Thanks for the review @mnriem and @copilot — addressed all three points in 74dcaf3:

1. Centralized path constant — added a module-level INTEGRATION_JSON = ".specify/integration.json" in workflows/engine.py. I kept it local to the package rather than importing specify_cli.INTEGRATION_JSON directly, because the CLI module imports the workflows package, so reusing the existing constant would create a circular import. A short comment in the code points to the canonical definition so future-you knows to keep them in sync.

2. UnicodeDecodeError fallback — extended the except clause to catch UnicodeDecodeError alongside JSONDecodeError/OSError, so a corrupt or non-UTF8 .specify/integration.json falls back to the literal default instead of crashing the engine. Added a regression test that writes \xff\xfe\x00\x00 (invalid UTF-8 leading byte) and confirms the workflow input resolves to "auto" without exception.

3. requires.integrations accuracy — dropped the narrow any: ["copilot", "claude", "gemini"] allowlist instead of broadening it. The four commands the workflow runs (speckit.specify, speckit.plan, speckit.tasks, speckit.implement) are core spec-kit commands provided by every integration in src/specify_cli/integrations/ (opencode, codex, qwen, cursor_agent, etc. — 30+ today), so the previous list was always inaccurate, not just after this PR. Replaced it with a comment noting the workflow runs against whichever integration the project was initialized with.

All 131 workflow tests pass locally.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 3

Comment thread workflows/speckit/workflow.yml Outdated
Comment on lines +9 to +13
requires:
speckit_version: ">=0.7.2"
integrations:
any: ["copilot", "claude", "gemini"]
# The four commands used below (specify, plan, tasks, implement) are core
# spec-kit commands provided by every integration, so this workflow runs
# against whichever integration the project was initialized with.
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requires.integrations constraint was removed here, but the PR description only mentions changing the integration input default/prompt. If this removal is intentional, please update the requires metadata to explicitly reflect the intended compatibility (e.g., broaden the allowed integrations) rather than dropping the field entirely, since other tooling/docs may rely on it as a declarative compatibility signal.

Copilot uses AI. Check for mistakes.
Comment thread src/specify_cli/workflows/engine.py Outdated
Comment thread src/specify_cli/workflows/engine.py Outdated
)
elif "default" in input_def:
resolved[name] = input_def["default"]
resolved[name] = self._resolve_default(name, input_def["default"])
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_resolve_inputs() coerces/validates provided values via _coerce_input(), but defaults (including the newly project-resolved integration default) bypass type coercion and enum validation. This can yield invalid inputs values even when an input definition declares an enum/type. Consider running the resolved default through _coerce_input() as well so defaults are validated consistently with user-provided inputs.

Suggested change
resolved[name] = self._resolve_default(name, input_def["default"])
default_value = self._resolve_default(name, input_def["default"])
resolved[name] = self._coerce_input(
name, default_value, input_def
)

Copilot uses AI. Check for mistakes.
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented May 1, 2026

Please address Copilot feedback. If not applicable, please explain why

Quratulain-bilal added a commit to Quratulain-bilal/spec-kit that referenced this pull request May 1, 2026
- Move INTEGRATION_JSON to a dedicated specify_cli._paths module so the
  CLI entrypoint and the workflows engine share a single source of
  truth instead of each carrying its own constant. specify_cli still
  re-exports INTEGRATION_JSON for backward compatibility.
- Run the resolved default value through _coerce_input() so workflow
  defaults (including the project-resolved integration default) are
  validated against declared type/enum constraints, just like
  user-provided inputs. Adds two regression tests covering an enum
  violation and numeric coercion of a string default.
- Restore requires.integrations as an explicit declarative compatibility
  signal, using the wildcard form ``any: "*"`` to accurately reflect
  that the workflow runs against any registered integration.
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

Round 2 — addressed in e9d946e:

1. Shared path constant (no duplication) — moved INTEGRATION_JSON to a new src/specify_cli/_paths.py module. Both specify_cli/__init__.py and specify_cli/workflows/engine.py now import from there, so there is exactly one source of truth. specify_cli.INTEGRATION_JSON is preserved as a re-export so external callers don't break. I went with a real shared module instead of the lazy __import__ pattern from the suggestion, since it avoids the import-time fallback path entirely and reads more straightforwardly.

2. Defaults now run through _coerce_input — applied the suggested change to _resolve_inputs, so resolved defaults (including the project-resolved integration default) are validated against declared type and enum constraints. Added two regression tests: one asserts a default that violates the enum raises ValueError, the other asserts a numeric default given as a string ("3") is coerced to int.

3. requires.integrations restored as a declarative signal — added it back as any: "*", the wildcard form, with a comment explaining that the four core commands (specify/plan/tasks/implement) are provided by every integration. This keeps the field in place as a compatibility hint for tooling/docs while accurately representing the workflow's actual reach (all 30+ registered integrations, not just copilot/claude/gemini).

Test results: 133 passed in tests/test_workflows.py; full suite 1732 passed, 125 skipped (no regressions from the __init__.py re-export change).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 5

Comment thread src/specify_cli/workflows/engine.py
Comment thread src/specify_cli/workflows/engine.py
Comment thread tests/test_workflows.py
Comment thread workflows/speckit/workflow.yml Outdated
Comment thread tests/test_workflows.py Outdated
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented May 1, 2026

Please address Copilot feedback. If not applicable, please explain why

Quratulain-bilal added a commit to Quratulain-bilal/spec-kit that referenced this pull request May 1, 2026
- workflows/speckit/workflow.yml: replace the wildcard ``any: "*"`` with
  an explicit list ``[claude, copilot, gemini, opencode]`` matching the
  documented schema for ``requires.integrations.any``. Comment notes the
  list is a non-exhaustive compatibility hint.
- engine.validate_workflow: validate input defaults at authoring time so
  enum/type mismatches in workflow YAML surface during install/validation
  instead of at execution. The ``integration: auto`` sentinel is exempt
  because it is resolved against project state at runtime.
- Extract a shared low-level helper ``try_read_integration_json`` into
  ``specify_cli._paths`` and use it from the workflow engine. Keeps
  parsing rules consistent between the CLI's loud loader and the engine's
  silent loader; CLI ``_read_integration_json`` retains its diagnostic
  semantics layered on top of the same parsing surface.
- Drop the now-unused INTEGRATION_JSON import in engine.py.
- Update test docstring to reflect actual engine behavior on missing
  integration state (no special-cased dispatcher error message exists).
- Add 2 regression tests for validate_workflow's new default checks.
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

Round 3 — addressed in 67e4d1f. Walking through each Copilot point:

1. requires.integrations.any: "*" not in documented schema — fair catch; replaced with an explicit list [claude, copilot, gemini, opencode] matching the documented schema. A short comment notes the list is a non-exhaustive compatibility hint and the workflow's four core commands run against any registered integration.

2. _resolve_inputs validates at runtime, not install time — added default validation to validate_workflow(). Authoring mistakes (e.g. a default not in the declared enum, or a non-numeric default for a type: number input) now surface during specify workflow validate/install instead of waiting for execution. The integration: auto sentinel is explicitly exempt because it is resolved against project state at runtime, not against the input's enum. Two new regression tests cover both the rejection path and the auto exemption.

3. _load_project_integration duplicates _read_integration_json parsing — extracted a shared low-level try_read_integration_json(project_root) -> dict | None into specify_cli._paths. The workflow engine now delegates to it; the CLI's loud _read_integration_json (which exits the process with diagnostics) keeps its existing per-exception messaging and is documented as the layered counterpart. This way both paths share the same parsing rules without forcing the CLI to drop its specific error reporting.

4. Test docstring overstates dispatcher behavior — applied the suggested wording change. The docstring now accurately states that the engine must not invent an integration; downstream resolution handles an unresolved "auto".

5. PR description test count drifted — updated the PR body to match the actual coverage (8 new regression tests, full breakdown in the "Fix" section).

Test results: 135 passed in tests/test_workflows.py; full suite still 1732 passed / 125 skipped.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/specify_cli/workflows/engine.py:741

  • _resolve_inputs() always runs resolved defaults through _coerce_input(). For integration: auto with an enum that does not include "auto", if .specify/integration.json is missing/malformed then _resolve_default() returns the literal "auto" and _coerce_input() will raise a ValueError during input resolution. This is inconsistent with the validation-time exemption for integration: auto and with the intended "fall back to literal default" behavior. Consider exempting enum validation for the "auto" sentinel at runtime (or skipping coercion until after auto has resolved to a concrete integration).
            elif "default" in input_def:
                default_value = self._resolve_default(name, input_def["default"])
                resolved[name] = self._coerce_input(name, default_value, input_def)
            elif input_def.get("required", False):
  • Files reviewed: 5/5 changed files
  • Comments generated: 1

Comment thread src/specify_cli/workflows/engine.py Outdated
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented May 1, 2026

Please address Copilot feedback. If not applicable, please explain wh

Quratulain-bilal added a commit to Quratulain-bilal/spec-kit that referenced this pull request May 1, 2026
…#2421

Previously the ``integration: auto`` exemption skipped default validation
entirely, which meant a workflow could declare an incompatible type
(e.g. ``type: number`` with ``default: "auto"``) and still pass
validation, only to fail later at runtime.

Both the install-time check in ``validate_workflow()`` and the runtime
coercion in ``_resolve_inputs()`` now exempt only the enum-membership
check for the auto sentinel; the declared type is still enforced. Adds
a regression test that asserts a ``type: number`` + ``default: "auto"``
workflow is rejected at validation time.
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

Round 4 — addressed in 0dd3426. Good catch.

Tightened the auto sentinel exemption to enum-only. Previously both the install-time validate_workflow() check and the runtime _resolve_inputs() coercion skipped all default validation when integration: auto — which meant a workflow could declare type: number with default: "auto" and slip through validation, only to fail later at runtime. Now both call sites build a derived input_def that drops just the enum key for the auto case (when an enum is present), and the declared type is still enforced.

Added a regression test that asserts a workflow declaring integration as type: number with default: "auto" is rejected at validation time with an "invalid default" error.

Test results: 136 passed in tests/test_workflows.py.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/specify_cli/init.py:1901

  • _read_integration_json() still re-implements JSON parsing directly. Since this PR introduces try_read_integration_json() as the shared low-level loader (including handling UnicodeDecodeError for non-UTF8 files), consider using that helper here (and layering the CLI’s user-facing diagnostics on top) to avoid drift and to ensure non-UTF8 integration.json doesn’t crash the CLI with an uncaught decode error.
from ._paths import INTEGRATION_JSON  # noqa: E402  (re-export for backward compat)


def _read_integration_json(project_root: Path) -> dict[str, Any]:
    """Load ``.specify/integration.json``.  Returns ``{}`` when missing."""
  • Files reviewed: 5/5 changed files
  • Comments generated: 1

Comment thread workflows/speckit/workflow.yml Outdated
Quratulain-bilal added a commit to Quratulain-bilal/spec-kit that referenced this pull request May 1, 2026
…thub#2421

- workflows/speckit/workflow.yml: bump ``requires.speckit_version`` from
  ``>=0.7.2`` to ``>=0.8.3`` so older spec-kit versions, which lack the
  engine-side resolution of the ``integration: "auto"`` sentinel, do not
  pull this workflow from the catalog and then fail by treating "auto"
  as a literal integration key. Adds an inline comment explaining the
  reason for the floor.
- src/specify_cli/__init__.py: ``_read_integration_json`` now also
  catches ``UnicodeDecodeError`` (non-UTF8 file) so the CLI fails with
  a targeted, actionable diagnostic instead of an uncaught traceback,
  and notes in the docstring that the function shares its low-level
  parsing surface with ``_paths.try_read_integration_json`` while
  keeping loud per-cause messaging on top.
@mnriem mnriem requested a review from Copilot May 1, 2026 16:08
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

Round 5 — addressed in ae4c94d.

1. requires.speckit_version floor (workflow.yml:10) — bumped from >=0.7.2 to >=0.8.3. The current development version is 0.8.3.dev0 (per pyproject.toml), so 0.8.3 is the first stable release that will ship with engine-side resolution of the integration: "auto" sentinel. Older versions would treat "auto" as a literal integration key at dispatch time. Added an inline comment in the YAML explaining the reason for the floor.

2. _read_integration_json could crash on non-UTF8 (low-confidence comment) — fair point; addressed by adding a UnicodeDecodeError clause to the CLI loader's exception chain, mirroring the workflow engine's tolerance. The CLI keeps its loud per-cause diagnostics (separate messages for invalid JSON, non-UTF8, OS errors, and non-object payloads) so users get an actionable hint pointing at .specify/integration.json. The docstring now notes the shared parsing surface with _paths.try_read_integration_json for future maintainers.

I deliberately kept _read_integration_json as a direct parser rather than layering on top of try_read_integration_json because the CLI's per-exception messaging is more useful to a human user than a single "could not load" fallback — the helper is the right shape for the silent engine path, and the CLI is the right shape for the loud user-facing path. Both now share the same set of caught exceptions, which addresses the drift concern.

Test results: 136 passed in tests/test_workflows.py.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 0 new

@Quratulain-bilal Quratulain-bilal force-pushed the fix/workflow-integration-auto-default branch from ae4c94d to 2886441 Compare May 2, 2026 08:05
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

happy to address any further feedback whenever you have a chance to review. CI is green and all review threads are resolved.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 1

Comment thread src/specify_cli/__init__.py Outdated
Comment on lines +1939 to +1943
except UnicodeDecodeError as exc:
console.print(f"[red]Error:[/red] {path} is not valid UTF-8.")
console.print(f"Please fix or delete {INTEGRATION_JSON} and retry.")
console.print(f"[dim]Details:[/dim] {exc}")
raise typer.Exit(1)
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented May 4, 2026

Please addres Copilot feedback. If not applicable, please explain why

Address Copilot feedback: UnicodeDecodeError can be raised by both
read_text() and json.loads(), so combining the handlers ensures both
cases produce a consistent, clear error message.
@Quratulain-bilal
Copy link
Copy Markdown
Contributor Author

@mnriem! Addressed Copilot feedback in 8c4488a combined the JSONDecodeError and
UnicodeDecodeError handlers since path.read_text(encoding="utf-8") can raise UnicodeDecodeError before
json.loads() runs, making the separate Unicode block unreachable. Single combined handler now covers

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 2

Comment on lines +797 to +799
data = json.loads(path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError, UnicodeDecodeError):
return None
Comment thread tests/test_workflows.py
Comment on lines +1499 to +1505
"""`integration: auto` should resolve to .specify/integration.json's integration."""
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition

specify_dir = project_dir / ".specify"
specify_dir.mkdir(parents=True, exist_ok=True)
(specify_dir / "integration.json").write_text(
json.dumps({"integration": "opencode", "version": "0.7.4"}),
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented May 5, 2026

Please address Copilot feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Workflow engine hardcodes "copilot" default, ignoring project's initialized integration (e.g., opencode)

3 participants