Skip to content

Rust SDK: bundle Copilot CLI by default#1385

Merged
SteveSandersonMS merged 5 commits into
mainfrom
stevesandersonms/rust-embedded-cli-default-on
May 22, 2026
Merged

Rust SDK: bundle Copilot CLI by default#1385
SteveSandersonMS merged 5 commits into
mainfrom
stevesandersonms/rust-embedded-cli-default-on

Conversation

@SteveSandersonMS
Copy link
Copy Markdown
Contributor

@SteveSandersonMS SteveSandersonMS commented May 22, 2026

Flips the Rust SDK's embedded-CLI experience to on by default, matching the .NET SDK's NuGet bundling behaviour. Consumers who just want a working binary now get one with no configuration; consumers who want a smaller binary opt out via default-features = false.

The companion goal was to remove the developer ceremony around the existing design (which required setting COPILOT_CLI_VERSION at consumer build time). After this PR, the CLI version and per-platform SHA-256 hashes are snapshotted from nodejs/package-lock.json automatically at SDK publish time, mirroring how the .NET SDK's _GenerateVersionProps BeforeTargets="Pack" writes GitHub.Copilot.SDK.props before NuGet packing.

What changed

Cargo features:

  • Renamed the cargo feature from embedded-cli (opt-in) to bundled-cli (opt-out). It is now in the default feature set. Consumers opt out via default-features = false (the closest Cargo-native analog of .NET's CopilotSkipCliDownload=true).
  • Archive helpers are target-conditional runtime deps: tar + flate2 on Linux/macOS, zip on Windows.
  • Dropped sha2 + zstd from runtime deps. The build-time SHA-256 verification still happens (against the snapshotted hash). Once embedded, the bytes are part of the consumer's signed binary and trusted.

Build-time pipeline:

  • build.rs now embeds the raw .tar.gz / .zip from GitHub Releases via include_bytes!() and no longer extracts or re-compresses with zstd. The recompress was the dominant build cost (~30s per build); skipping it makes warm rebuilds significantly faster.
  • build.rs resolves the version and hash from one of three sources, in order:
    1. COPILOT_CLI_VERSION env var (advanced override, fetches live SHA256SUMS.txt).
    2. bundled_cli_version.txt snapshot file (published-crate consumer).
    3. Sibling ../nodejs/package-lock.json + live SHA256SUMS.txt (mono-repo contributor build).
  • Build-time safety net: build.rs probes the downloaded archive to confirm copilot[.exe] is present at the expected path. If the upstream layout ever changes, the build fails loudly rather than shipping a broken bundle.
  • Visible cargo:warning=Downloading <url> on cache miss; silent on cache hit. Cargo's only mechanism for warning-level build script output is cargo:warning=; there is no info channel.

Version pinning (no developer ceremony):

  • rust/scripts/snapshot-bundled-cli-version.sh reads nodejs/package-lock.json and the matching release's SHA256SUMS.txt, then writes rust/bundled_cli_version.txt.
  • The publish-rust job in .github/workflows/publish.yml invokes the script between "Set version" and "Package (dry run)", with a verification step that fails the publish if the file was not generated.
  • The snapshot file is gitignored locally. Cargo's include / exclude are mutually exclusive, so Cargo.toml migrates from exclude to include so the gitignored file ships in published tarballs. The previously-excluded files (RELEASING.md, the rustfmt configs, etc.) are not matched by the new include patterns and continue to be filtered out.

Resolution surface (reduced):

  • Removed PATH scanning and all the homebrew / nvm / fnm / volta / nodenv walks (~280 lines deleted from resolve.rs).
  • Resolution order now matches the .NET and TypeScript SDKs: explicit CliProgram::Path > COPILOT_CLI_PATH > bundled CLI > Error::BinaryNotFound. No silent PATH fallback.
  • Demoted the resolve and embeddedcli modules to pub(crate). They were public introspection helpers with no C# / TypeScript equivalent. Client::start is now the only consumer.
  • Dropped the unused embeddedcli::bundled_version() public helper.

App-developer override for the runtime extraction location:

  • New public ClientOptions::bundled_cli_extract_dir: Option<PathBuf> field plus a matching with_bundled_cli_extract_dir(PathBuf) builder.
  • embeddedcli::install() was split so an internal pub(crate) install_at(dir) helper can target an arbitrary directory; Client::start threads the override through.
  • Unset (default) behaviour is unchanged: lazy extraction to the platform cache dir from dirs::cache_dir() (%LOCALAPPDATA% on Windows, ~/Library/Caches/ on macOS, $XDG_CACHE_HOME or ~/.cache/ on Linux), under github-copilot-sdk-{version}/.

README rewrite: new "Embedded CLI" section explains the default-on bundling, the default-features = false opt-out, the publish-time snapshot mechanism, and with_bundled_cli_extract_dir. Features table updated.

What did NOT change

  • Default runtime extraction location (still dirs::cache_dir()).
  • Download source (still GitHub Releases, still copilot-{platform}.{tar.gz,zip}).

Validation

End-to-end test against a real Copilot CLI: built a fresh consumer crate in a temp directory that pulled the SDK via a path dependency, ran cargo build (visible Downloading https://.../copilot-win32-x64.zip warning), then ran the resulting binary which extracted the CLI in ~1.9s, created a session, sent a message, and got the expected reply. Confirms the full pipeline works default-on with no consumer configuration.

Other validation:

  • cargo check --features test-support (bundled path) - green
  • cargo check --no-default-features (opt-out path) - green
  • cargo check --tests --features test-support - green
  • cargo clippy --tests --features test-support -- -D warnings - green
  • cargo fmt --check - green
  • cargo fmt --check against the nightly .rustfmt.nightly.toml - green
  • cargo package --list --allow-dirty confirms bundled_cli_version.txt is included when present

Reviewer attention

  • Cargo.toml's include list: please confirm no source patterns are missing relative to today's default plus the previously-excluded files we wanted to keep out.
  • publish.yml: the snapshot + verify steps are bash-only and assume node + curl are available on the ubuntu-latest runner (they are).
  • rust-sdk-tests.yml: the main cargo test job now runs with --no-default-features --features test-support so it does not pull the 130 MB CLI archive every run. The dedicated cross-platform bundle job still exercises the bundling pipeline.

SteveSandersonMS and others added 2 commits May 22, 2026 19:51
Flips the embedded-CLI experience to on-by-default with an opt-out
cargo feature, matching the .NET SDK's NuGet bundling behavior.

Build-time
----------

* `bundled-cli` cargo feature replaces `embedded-cli`. It is now in the
  default feature set, so `cargo add github-copilot-sdk` plus default
  Cargo.toml settings ships a self-contained CLI binary inside the
  consumer's compiled artifact. Consumers opt out via
  `default-features = false` (the closest Cargo-native analog of .NET's
  `CopilotSkipCliDownload=true`).

* build.rs no longer requires `COPILOT_CLI_VERSION` at consumer build
  time. It resolves the version + per-platform SHA-256 hashes from one
  of three sources, in order:

  1. `COPILOT_CLI_VERSION` env-var override (advanced; fetches live
     SHA256SUMS.txt from GitHub Releases).
  2. A `bundled_cli_version.txt` snapshot file at the crate root. The
     file is gitignored locally and ships only in published crate
     tarballs; the rust-crate publish workflow regenerates it on every
     publish from `nodejs/package-lock.json` + the matching release's
     SHA256SUMS.txt.
  3. Sibling `../nodejs/package-lock.json` plus live SHA256SUMS.txt
     (the mono-repo contributor path; identical to today's behavior).

* `rust/scripts/snapshot-bundled-cli-version.sh` performs the snapshot.
  The `publish-rust` job in `.github/workflows/publish.yml` invokes it
  between "Set version" and "Package (dry run)", with a follow-up
  verification step that fails the publish if the file was somehow not
  generated.

* `Cargo.toml` migrates from `exclude = [...]` to `include = [...]` so
  the gitignored snapshot file is included by `cargo publish`. Cargo's
  include/exclude are mutually exclusive; today's exclude entries
  (RELEASING.md, rustfmt configs, etc.) are not matched by the new
  include patterns and therefore continue to be filtered out of the
  published tarball.

Runtime
-------

* `embeddedcli::path()` continues to lazy-extract to
  `~/.cache/github-copilot-sdk-{version}/copilot[.exe]` and SHA-verify
  on first use. No change for the common case.

* New `ClientOptions::with_bundled_cli_extract_dir(PathBuf)` builder +
  matching public field let app developers redirect the bundled-CLI
  extraction (e.g. to a CI-scratch directory) without changing the
  global cache layout. Implemented via a `pub(crate)` install_at helper
  on `embeddedcli`; `Client::start` threads the override through the
  resolve step.

* When the `bundled-cli` feature is disabled, `embeddedcli::path()` and
  `bundled_version()` return `None`, the runtime sha2/zstd deps are not
  in the crate graph at all, and resolve falls back to PATH-based
  binary lookup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bundled-cli feature is now the default-on cargo feature. The Rust SDK Tests rust-bundled-cli matrix job simplifies to just cargo build (default features). COPILOT_CLI_VERSION env var is no longer required because build.rs auto-reads the sibling nodejs/package-lock.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SteveSandersonMS SteveSandersonMS marked this pull request as ready for review May 22, 2026 19:12
@SteveSandersonMS SteveSandersonMS requested a review from a team as a code owner May 22, 2026 19:12
Copilot AI review requested due to automatic review settings May 22, 2026 19:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Rust SDK to bundle the Copilot CLI by default, aligning the Rust crate UX with the .NET SDK’s “it just works” bundling experience. It introduces a publish-time snapshot mechanism to pin the CLI version/hashes without requiring consumers to set build-time environment variables, and adds an option to override the runtime extraction directory.

Changes:

  • Switch Cargo features to bundled-cli (enabled by default) and gate runtime deps (sha2, zstd) behind it.
  • Add publish-time CLI version/hash snapshotting (bundled_cli_version.txt) and update build.rs to resolve version/hash from env override, snapshot file, or monorepo lockfile.
  • Add ClientOptions::bundled_cli_extract_dir to control where the bundled CLI is extracted at runtime.
Show a summary per file
File Description
rust/src/resolve.rs Threads an optional extraction directory into bundled-CLI resolution.
rust/src/lib.rs Exposes bundled_cli_extract_dir on ClientOptions and plumbs it into Client::start.
rust/src/embeddedcli.rs Splits installation to support installing into a caller-specified directory.
rust/scripts/snapshot-bundled-cli-version.sh New publish-time script to snapshot CLI version + SHA-256 hashes into bundled_cli_version.txt.
rust/README.md Documents default-on bundling, opt-out via default-features = false, and extraction override.
rust/Cargo.toml Switches to include = [...] packaging and introduces default bundled-cli feature.
rust/build.rs Resolves bundled CLI version/hash from env/snapshot/lockfile; bundles by default when feature enabled.
rust/.gitignore Ignores the generated bundled_cli_version.txt locally.
.github/workflows/rust-sdk-tests.yml Updates CI commentary and bundling smoke-test steps; adjusts archive caching.
.github/workflows/publish.yml Adds snapshot + verification steps before Rust packaging/publishing.

Copilot's findings

  • Files reviewed: 10/10 changed files
  • Comments generated: 5

Comment thread rust/scripts/snapshot-bundled-cli-version.sh Outdated
Comment thread rust/src/lib.rs Outdated
Comment thread .github/workflows/rust-sdk-tests.yml
Comment thread .github/workflows/rust-sdk-tests.yml Outdated
Comment thread rust/src/embeddedcli.rs Outdated
SteveSandersonMS and others added 3 commits May 22, 2026 21:46
Build-time

* �uild.rs no longer extracts the binary or re-compresses it with zstd. Instead it embeds the raw .tar.gz / .zip from GitHub Releases. The Downloading message is the only build-time output (no Caching log on cache hit). Build.rs still SHA-256 verifies the download and probes the archive layout to confirm the binary entry exists - if upstream changes layout, the build fails loudly rather than shipping a broken bundle.

* Removed sha2 + zstd runtime deps. The bytes are part of the consumer's signed binary by the time we extract; further hashing adds no value. 	ar + late2 (Linux/macOS) and zip (Windows) are now target-conditional optional runtime deps gated on �undled-cli. Build-deps stay unconditional so cross-compilation works.

Runtime

* embeddedcli::install() now archive-extracts at first call rather than zstd-decompressing. Per-version install dir means we can skip extraction when the target file already exists. About 1.9s on Windows for a fresh extract.

Resolution surface

* Dropped PATH scanning + all the homebrew/nvm/fnm/volta/nodenv walks (~280 lines removed from resolve.rs).

* Kept the .NET/TypeScript-equivalent resolution order: CliProgram::Path > COPILOT_CLI_PATH > bundled CLI > error. No silent fallback.

* Demoted 
esolve + embeddedcli modules and their contents to pub(crate). They were public introspection helpers with no C#/.NET equivalent. Client::start is now the only consumer.

* dirs runtime dep is now gated on �undled-cli (only used by the default install-dir resolution).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Reword doc references from \~/.cache/...\ (Linux-specific) to the platform cache dir returned by dirs::cache_dir() (Windows: %LOCALAPPDATA%, macOS: ~/Library/Caches/, Linux: \ or ~/.cache). Hits embeddedcli.rs, lib.rs ClientOptions doc, and README.

* Restore CLI version in the bundle-cache key in rust-sdk-tests.yml so old archives drop out when the pinned version bumps (was OS-only, would have accumulated indefinitely).

* Main \cargo test\ job now runs with \--no-default-features --features test-support\ so it doesn't pull the 130MB CLI archive on every test run (the dedicated bundle job already exercises that pipeline; tests use COPILOT_CLI_PATH from setup-copilot).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SteveSandersonMS SteveSandersonMS merged commit a68141d into main May 22, 2026
24 checks passed
@SteveSandersonMS SteveSandersonMS deleted the stevesandersonms/rust-embedded-cli-default-on branch May 22, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants