Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions content/blog/2026-05-05-kotlin-as-reference-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ Then `make plugins-update && make check`. The build pipeline (Story 13.4) render

We picked Kotlin as the reference extraction because:

1. **It's the freshest core language.** Added in March 2026; the install logic is recent and well-tested. Lower risk of "I forget what this Dockerfile bit does."
2. **Its install script is the heaviest in the matrix.** ktlint, detekt, and gradle each download from a separate upstream. JDK 21 from a builder stage. If extraction works for Kotlin, it works for almost anything.
1. **It's the most recent core language.** Added in v1.8 (March 2026), so the install logic is fresh in the maintainers' minds.
2. **Its install script pulls from multiple upstreams.** ktlint and detekt are downloaded binaries; gradle is a separate distribution; JDK 21 comes from a builder stage. The variety stresses the manifest's `copy_from_builder` + `install_script` contract more than a simpler language would.
3. **The `copy_from_builder` pattern is non-trivially exercised.** A whole JDK tree gets COPY'd from `eclipse-temurin:21-jdk`. The plugin manifest reproduces that exactly without dev-toolchain having to know about Kotlin.

## The extraction recipe
Expand All @@ -55,26 +55,26 @@ The full text is [here](https://github.com/devrail-dev/devrail-standards/blob/ma

## What's regression-tested

Plus the manifest-shape side: dev-toolchain ships a new smoke test in `tests/test-kotlin-plugin-extraction.sh` that:
On the manifest-shape side, dev-toolchain ships a new smoke test in `tests/test-kotlin-plugin-extraction.sh` that:

1. Validates `devrail-plugin-kotlin/plugin.devrail.yml` against schema_version 1.
2. Resolves the plugin via `file://` URL from a vendored fixture (`tests/fixtures/kotlin-via-plugin/`) and confirms `.devrail.lock` records the resolved SHA + content_hash.
3. Loads the plugin into the dispatcher cache and asserts name / version / devrail_min_version match.
4. Walks every target (lint / format_check / format_fix / test / security) and confirms the cmd + gate shape parity with the in-core HAS_KOTLIN behaviour. Specific assertions on `ktlint && detekt-cli` chaining catch regressions where someone changes the manifest but forgets to keep both tools wired up.

The full docker-build of `devrail-local:<hash>` with real ktlint/detekt/gradle downloads (~5 minutes) is NOT in CI — it's a maintainer-run manual check, same trade-off we made for the `minimal-v1` fixture in v1.10. CI gets a fast hermetic regression; humans do the heavy validation.
The full docker-build of `devrail-local:<hash>` with real ktlint / detekt / gradle / JDK downloads is NOT in CI — it's a maintainer-run manual check, same trade-off we made for the `minimal-v1` fixture in v1.10. The build is heavy (multi-stage docker build + JVM-binary downloads + extracting a Gradle distribution); CI gets a fast hermetic regression instead, and humans do the heavy validation.

## What's coming in v2.0.0

The reason this extraction is additive is that we've committed to back-compat through v1.x. Existing consumers with `languages: [kotlin]` should not have to do anything when v1.11 lands. Their `make check` runs unchanged.

v2.0.0 (Story 13.9, no scheduled date yet) flips the model:
v2.0.0 (Story 13.9, no scheduled date yet) flips the model: it removes ALL `HAS_<LANG>` blocks and per-language Dockerfile bits from `dev-toolchain` core. Every language becomes plugin-based. The plan as it stands today:

- Remove the in-core `HAS_KOTLIN` blocks and Kotlin Dockerfile bits. Same for every other core language as we extract them.
- Ship `devrail-init migrate --to v2` to walk consumer `.devrail.yml` files: any `languages:` entry that's no longer in core gets moved to `plugins:` with the appropriate plugin source pinned.
- Retire the in-core `HAS_<LANG>` paths in one cliff at the major bump.
- Ship `devrail-init migrate --to v2` to walk consumer `.devrail.yml` files and rewrite `languages:` entries as `plugins:` references with appropriate sources pinned.
- Major version bump signals the breaking change.

Between now and then, every other core language gets the same extraction treatment (Swift, Ruby, Go, etc.). Each will follow the recipe documented from this Kotlin pass. We'll likely ship them as separate v1.x minor releases so the migration burden is incremental rather than one big v2 cliff.
How (and how many of) the other core languages get extracted into reference plugins between now and then is still being scoped. Story 13.7 covered Kotlin only; we'll learn from the Kotlin extraction before committing to a fixed schedule for Swift, Ruby, Go, and the rest.

## Try it

Expand Down
11 changes: 5 additions & 6 deletions content/blog/2026-05-05-plugin-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ date: 2026-05-05
description: "DevRail v1.10 introduces a plugin architecture so anyone can ship a new language or tool integration without forking dev-toolchain. Loader, lockfile, extended-image build, and execution dispatch all in one container, one make check."
---

For the first eighteen months of DevRail, every new language meant a PR against `dev-toolchain` -- a Dockerfile change, an install script, Makefile blocks for `_lint` / `_format` / `_test` / `_security`, a standards doc, and a release. That worked while we were stabilizing the eight core ecosystems (Python, Bash, Terraform, Ansible, Ruby, Go, JavaScript/TypeScript, Rust, and most recently Swift and Kotlin), but it doesn't scale to the long tail of languages and tools real teams use.
Up to now, every new language in DevRail has meant a PR against `dev-toolchain` -- a Dockerfile change, an install script, Makefile blocks for `_lint` / `_format` / `_test` / `_security`, a standards doc, and a release. That worked through the ten core ecosystems we shipped from MVP through v1.9 (Python, Bash, Terraform, Ansible, Ruby, Go, JavaScript/TypeScript, Rust, Swift, and Kotlin), but it doesn't scale to the long tail of languages and tools real teams use.

**v1.10.6 ships a plugin architecture.** Anyone can now publish a `devrail-plugin-<name>` git repo, and any DevRail-managed project can declare it in `.devrail.yml` and pick up new tools at the next `make check`. No fork, no PR, no waiting on a release. The "one container, one make check" guarantee holds throughout.

Expand All @@ -29,20 +29,19 @@ plugins:

1. **Loader (Story 13.2)** validates `plugin.devrail.yml` against schema_version 1, enforcing `devrail_min_version` and per-target shape.
2. **Resolver + lockfile (Story 13.3)** resolves `rev:` to an immutable SHA, fetches the plugin tree to a content-addressed cache, and records the resolved metadata in `.devrail.lock`. Branch refs are rejected. Tag-rebases are detected via content_hash mismatch.
3. **Extended-image build (Story 13.4)** generates a project-local `Dockerfile.devrail` that layers each plugin's apt / COPY / ENV / install_script onto `ghcr.io/devrail-dev/dev-toolchain:v1`, then builds `devrail-local:<hash-of-dockerfile>` via BuildKit. Cache hits are free; first builds take 30 s -- 2 min depending on the plugin.
3. **Extended-image build (Story 13.4)** generates a project-local `Dockerfile.devrail` that layers each plugin's apt / COPY / ENV / install_script onto `ghcr.io/devrail-dev/dev-toolchain:v1`, then builds `devrail-local:<hash-of-dockerfile>` via BuildKit. Cache hits are instant; first builds depend entirely on the plugin's install script (whatever ktlint / cargo / npm / etc. take to run plus the cost of any apt packages or `copy_from_builder` payloads).
4. **Execution loop (Story 13.5)** dispatches each plugin's matching target inside the existing `_lint` / `_format` / `_fix` / `_test` / `_security` recipes, with gate evaluation, `{paths}` interpolation, per-language overrides, and JSON aggregation into the same envelope as core results. Consumers can't tell from the JSON output which results came from core and which from a plugin.

`DEVRAIL_FAIL_FAST=1` short-circuits on plugin failures the same as core. Workspaces without `plugins:` in `.devrail.yml` see byte-identical behavior to v1.9.x -- the loader writes an empty cache, the dispatcher exits immediately, no extra events.

## Why now

Three forces aligned:
Two reasons:

- **The core surface stabilized.** With ten languages shipped (the most recent two -- Swift and Kotlin -- landed in March) the patterns for "what goes in `_lint`, `_test`, etc." are clear enough to expose as a contract.
- **The container model is cheaper than people think.** BuildKit content-addresses every layer; an unchanged plugin set is an instant cache hit. We benchmarked Elixir + Rust + Swift in the same project and the second `make check` was within 200 ms of the first -- the entire build pipeline boils down to a `docker image inspect`.
- **Real teams have real tools we shouldn't ship.** Mojo. Zig. Roc. Crystal. Internal DSLs. Every one of these comes up in conversation; none of them belongs in `dev-toolchain` core. A plugin gives them a first-class home with the same UX as the languages we do ship.
- **Caching makes the container model cheap.** BuildKit content-addresses every layer; an unchanged plugin set is an instant cache hit -- on a cache hit, `make check`'s entire plugin overhead boils down to a `docker image inspect`.

The architecture is documented in detail in the [design doc on GitHub](https://github.com/devrail-dev/dev-toolchain/blob/main/docs/plugin-architecture.md). The TL;DR: we surveyed Terraform providers, GitHub Actions, pre-commit, and pip extras, then picked declarative YAML manifests + git-repo distribution + immutable refs + a single execution mode (extended container image). The single-mode choice is deliberate -- DevRail's value proposition is one container, one make check, and we kept it.
The architecture is documented in detail in the [design doc on GitHub](https://github.com/devrail-dev/devrail-standards/blob/main/_bmad-output/planning-artifacts/plugin-architecture-design.md). The TL;DR: we surveyed Terraform providers, GitHub Actions, and pre-commit hooks, then picked declarative YAML manifests + git-repo distribution + immutable refs + a single execution mode (extended container image). The single-mode choice is deliberate -- DevRail's value proposition is one container, one make check, and we kept it.

## Authoring a plugin

Expand Down
2 changes: 1 addition & 1 deletion content/docs/contributing/adding-a-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,5 +114,5 @@ These are deferred to later phases:
## Next steps

- Read the [canonical plugin authoring guide](https://github.com/devrail-dev/devrail-standards/blob/main/standards/contributing.md#contributing-a-plugin) for field-by-field details, container integration patterns, and the pre-publish checklist.
- Read the [plugin architecture design doc](https://github.com/devrail-dev/dev-toolchain/blob/main/docs/plugin-architecture.md) for the full rationale and lifecycle.
- Read the [plugin architecture design doc](https://github.com/devrail-dev/devrail-standards/blob/main/_bmad-output/planning-artifacts/plugin-architecture-design.md) for the full rationale and lifecycle.
- See the [`plugins:` schema documentation](https://github.com/devrail-dev/devrail-standards/blob/main/standards/devrail-yml-schema.md) for the consumer-side declaration shape.
Loading