Skip to content

Add nuget-trusted-publishing skill#210

Merged
ViktorHofer merged 3 commits into
mainfrom
add-nuget-trusted-publishing
Mar 5, 2026
Merged

Add nuget-trusted-publishing skill#210
ViktorHofer merged 3 commits into
mainfrom
add-nuget-trusted-publishing

Conversation

@lewing
Copy link
Copy Markdown
Member

@lewing lewing commented Mar 4, 2026

Add nuget-trusted-publishing skill

Knowledge-driven skill for setting up NuGet trusted publishing (OIDC) on GitHub Actions repos -- replacing long-lived API keys with short-lived tokens.

5-Phase Process

  1. Discovery -- detect packable projects, classify package type (library/tool/MCP/template)
  2. Structure Validation -- verify MSBuild properties and supporting files
  3. Local Pre-Publish Testing -- pack and test locally before configuring nuget.org
  4. nuget.org Policy Setup -- guide through trusted publishing policy + GitHub Environment creation
  5. Workflow Setup -- create or modify publish workflow with OIDC auth

Files

  • SKILL.md -- orchestrating skill (~1900 tokens)
  • references/package-types.md -- per-type structural requirements and detection logic
  • references/publish-workflow.md -- complete tag-triggered workflow template

Validated against

  • Real MCP server publish session (helix.mcp -- greenfield, environment secrets, server.json versioning)
  • Real dotnet tool publish session (dotnet-replay -- greenfield, GitHub Release conflicts, re-run gotchas)
  • Multi-model review (Sonnet 4, GPT-5)
  • Skill-builder best practices checklist

Replaces #34 which could not be reopened after repo history rewrite.

Knowledge-driven skill for setting up NuGet trusted publishing (OIDC)
on GitHub Actions repos, replacing long-lived API keys with short-lived tokens.

Replaces #34 which could not be reopened after history rewrite.
Copilot AI review requested due to automatic review settings March 4, 2026 20:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new knowledge-driven .NET skill intended to guide users through setting up NuGet trusted publishing (OIDC) for GitHub Actions, including reference material and an eval scenario suite.

Changes:

  • Adds the nuget-trusted-publishing skill content (SKILL.md) describing a phased setup process for NuGet OIDC/trusted publishing.
  • Adds reference docs covering package-type detection/requirements and a publish workflow template.
  • Adds skill-validator eval scenarios for greenfield setup, implicit OIDC request, and API-key migration.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/dotnet/skills/nuget-trusted-publishing/SKILL.md New skill instructions for setting up NuGet trusted publishing (OIDC) in GitHub Actions.
src/dotnet/skills/nuget-trusted-publishing/references/package-types.md Reference doc for detecting/validating different NuGet package types before publishing setup.
src/dotnet/skills/nuget-trusted-publishing/references/publish-workflow.md Reference publish workflow template using NuGet/login + id-token: write.
src/dotnet/tests/nuget-trusted-publishing/eval.yaml New eval scenarios intended to validate the skill’s guidance for trusted publishing and migration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/dotnet/skills/nuget-trusted-publishing/SKILL.md Outdated
Comment thread plugins/dotnet/skills/nuget-trusted-publishing/SKILL.md
Comment thread tests/dotnet/nuget-trusted-publishing/eval.yaml
Comment on lines +2 to +6
- name: "Set up trusted publishing for a new NuGet library"
prompt: |
I have a .NET class library at src/MyLib/MyLib.csproj that I want to publish
to nuget.org using trusted publishing (OIDC) instead of an API key.
The repo is hosted on GitHub at myorg/mylib. Help me set this up.
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These scenarios reference repo files/structure (e.g., src/MyLib/MyLib.csproj) but the eval has no setup: section. Skill-validator runs scenarios in a fresh temp directory, so the referenced project/workflow files won’t exist unless you create them. Consider adding setup.files for a minimal packable .csproj (and, for the migration scenario, a baseline publish workflow that uses secrets.NUGET_API_KEY) so the skill can actually inspect/modify the repo as instructed.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

@ViktorHofer ViktorHofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the folder structure. The skill should be under plugins not src. The test should be under the root tests folder.

@lewing
Copy link
Copy Markdown
Member Author

lewing commented Mar 4, 2026

Eval Results (Run 5 — fresh PR after repo history rewrite)

Scenario Baseline With Skill Δ Skills Loaded Overfit Verdict
Set up trusted publishing for a new NuGet library 3.0/5 4.7/5 +1.7 ✅ nuget-trusted-publishing; tools: skill, glob ✅ 0.11
Set up NuGet publishing without mentioning trusted publishing 2.3/5 4.3/5 +2.0 ✅ nuget-trusted-publishing; tools: report_intent, skill, glob, view, bash, create ✅ 0.11
Migrate existing workflow from API key to trusted publishing 2.7/5 4.7/5 +2.0 ✅ nuget-trusted-publishing; tools: skill, bash ✅ 0.11

Model: claude-opus-4.6 | Judge: claude-opus-4.6 | Runs: 3

All three scenarios pass with +1.7 to +2.0 improvement over baseline. No overfitting detected.

@lewing
Copy link
Copy Markdown
Member Author

lewing commented Mar 4, 2026

Please update the folder structure. The skill should be under plugins not src. The test should be under the root tests folder.

the previous version of the pr was broken by other changes you made, I'm in the process of restoring it.

…dd CODEOWNERS

- Move skill from src/dotnet/skills/ to plugins/dotnet/skills/nuget-trusted-publishing/
- Move eval from src/dotnet/tests/ to tests/dotnet/nuget-trusted-publishing/
- Fix Phase 3/4 sequencing conflict: clarify that drafting the workflow is OK
  before policy confirmation, but publishing/removing secrets should wait
- Add CODEOWNERS entry for nuget-trusted-publishing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 4, 2026

Skill Validation Results

Skill Scenario Baseline With Skill Δ Skills Loaded Overfit Verdict
nuget-trusted-publishing Set up trusted publishing for a new NuGet library 3.0/5 4.0/5 +1.0 ✅ nuget-trusted-publishing; tools: skill, edit, stop_bash ✅ 0.09
nuget-trusted-publishing Set up NuGet publishing without mentioning trusted publishing 2.0/5 4.3/5 +2.3 ✅ nuget-trusted-publishing; tools: skill, report_intent, glob, view ✅ 0.09
nuget-trusted-publishing Migrate existing workflow from API key to trusted publishing 2.3/5 4.7/5 +2.4 ✅ nuget-trusted-publishing; tools: skill, bash ✅ 0.09

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

@lewing lewing requested a review from ViktorHofer March 4, 2026 20:43
@ViktorHofer ViktorHofer merged commit f7abee2 into main Mar 5, 2026
7 checks passed
@ViktorHofer ViktorHofer deleted the add-nuget-trusted-publishing branch March 5, 2026 08:16
@danmoseley
Copy link
Copy Markdown
Member

Someone from Nuget team should sign off on this IMO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants