dotnet-ai-ml by jeffschwMSFT · Pull Request #177 · dotnet/skills

jeffschwMSFT · 2026-03-04T00:51:58Z

This is the updated PR for the AI SDK technology selection PR (sorry the previous PR/branch was lost in the private to public move)

jeffschwMSFT · 2026-03-04T00:55:22Z

Scenario	Improvement	Avg Per-Run Score	Skill Activated
ML.NET classification on tabular data	0.61	0.51	✅ dotnet-ai-ml
LLM integration with MEAI abstraction	0.56	0.31	✅ dotnet-ai-ml
Reject LLM for tabular classification	0.77	0.78	✅ dotnet-ai-ml
Agentic workflow with guardrails	0.14	0.31	✅ dotnet-ai-ml
Natural-language scenario decomposition — RAG chatbot	0.35	0.41	✅ dotnet-ai-ml
RAG pipeline with vector search	0.14	0.01	✅ dotnet-ai-ml

Overall: ✅ Passed | Model: claude-opus-4.6 | Overfitting: 🟡 0.25

Copilot

Pull request overview

Adds a new dotnet-ai plugin skill focused on AI/ML technology selection in .NET, along with an evaluation suite to validate the skill’s guidance across classic ML, LLM usage, agentic workflows, and RAG scenarios.

Changes:

Introduces plugins/dotnet-ai with plugin.json and a new technology-selection skill document.
Adds tests/dotnet-ai/technology-selection/eval.yaml with multiple scenarios/rubrics to evaluate technology-selection behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
tests/dotnet-ai/technology-selection/eval.yaml	New eval scenarios/assertions for dotnet-ai technology selection across ML.NET, MEAI, agents, and RAG.
plugins/dotnet-ai/skills/technology-selection/SKILL.md	New skill guidance/decision tree and implementation guardrails for .NET AI/ML tech selection.
plugins/dotnet-ai/plugin.json	Registers the new dotnet-ai plugin.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jeffschwMSFT · 2026-03-06T20:33:16Z

Scenario	Baseline	With Skill	Δ	Verdict
ML.NET classification on tabular data	2.3/5	5.0/5	+2.7	✅
LLM integration with MEAI abstraction	1.0/5	3.0/5	+2.0	✅
Reject LLM for tabular classification	3.0/5	4.7/5	+1.7	✅
Agentic workflow with guardrails	2.0/5	3.0/5	+1.0	✅
Natural-language scenario decomposition — RAG chatbot	4.0/5	4.3/5	+0.3	✅
RAG pipeline with vector search	3.3/5	5.0/5	+1.7	✅

Overall: ✅ Passed (6/6) | Model: claude-opus-4.6 | Overfitting: 🟡 0.32

jeffschwMSFT · 2026-03-09T17:10:59Z

@luisquintanilla thanks for chatting on Friday. my recommendation is to code review and merge this one and we can iterate on the updated skill in your other PR. I will share some feedback on those

ViktorHofer

Needs marketplace.json and README.md updates. See https://github.com/dotnet/skills/blob/main/CONTRIBUTING.md#plugin-organization

ManishJayaswal · 2026-03-10T03:33:08Z

@artl93 @joperezr @jeffhandley - please review

artl93

Approving on the basis that this will be rectified with @luisquintanilla's comments once merged as stated.

jeffschwMSFT · 2026-03-11T15:12:19Z

/evaluate

github-actions · 2026-03-11T15:27:14Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
technology-selection	ML.NET classification on tabular data	3.0/5 → 4.3/5 ⏰ 🟢	✅ technology-selection; tools: skill	🟡 0.33	✅
technology-selection	LLM integration with MEAI abstraction	1.0/5 → 2.3/5 🟢	✅ technology-selection; tools: skill	🟡 0.33	✅
technology-selection	Reject LLM for tabular classification	3.0/5 → 4.3/5 🟢	✅ technology-selection; tools: skill, task, read_bash, create	🟡 0.33	✅
technology-selection	Agentic workflow with guardrails	1.7/5 ⏰ → 1.0/5 ⏰ 🔴	✅ technology-selection; tools: skill, task, read_bash	🟡 0.33	❌
technology-selection	Natural-language scenario decomposition — RAG chatbot	3.7/5 → 4.0/5 🟢	✅ technology-selection; tools: skill	🟡 0.33	✅
technology-selection	RAG pipeline with vector search	4.0/5 → 4.7/5 🟢	✅ technology-selection; tools: skill	🟡 0.33	✅

⏰ timeout — run hit the scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

luisquintanilla

Looks great. Added a few suggestions around ML.NET determinism, numeric libraries and token counting

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

.claude-plugin/marketplace.json:1

The source paths are relative (./plugins/...) but this file lives under .claude-plugin/, so these paths will typically resolve to .claude-plugin/plugins/... (which likely doesn’t exist). If the consumer resolves paths relative to the file location, update these to ../plugins/... (and similarly for all entries) or otherwise align with whatever path-resolution convention the Claude plugin loader expects.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Keep both dotnet-maui (from main) and dotnet-ai (from this branch) in README.md, marketplace.json files, and CODEOWNERS. Restore .claude-plugin/marketplace.json as a real file (matching .github/plugin/marketplace.json) instead of a symlink. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-11T22:08:50Z

+<PackageReference Include="Microsoft.ML" Version="4.*" />
+<PackageReference Include="Microsoft.ML.AutoML" Version="0.*" />
+<!-- Only if custom numerical work is needed: -->
+PackageReference Include="System.Numerics.Tensors" Version="10.*"


The XML snippet under “Classic ML packages” has an invalid PackageReference line (missing the opening < and closing />). Since this is intended to be copy/pasteable, please fix the malformed System.Numerics.Tensors reference so the example is valid XML.

Suggested change

PackageReference Include="System.Numerics.Tensors" Version="10.*"

<PackageReference Include="System.Numerics.Tensors" Version="10.*" />

Copilot · 2026-03-11T22:08:50Z

+<!-- Cloud LLM provider (pick one) -->
+<PackageReference Include="Azure.AI.OpenAI" Version="2.*" />
+<!-- OR -->
+<PackageReference Include="OpenAI" Version="2.*" />


The package list and the DI example are inconsistent: the snippet shows services.AddChatClient(...).UseOpenAIChatClient(...), but the listed packages only include Microsoft.Extensions.AI plus raw provider SDKs (OpenAI / Azure.AI.OpenAI). Unless UseOpenAIChatClient is available from those packages (typically it comes from a provider-specific MEAI integration package), this example won’t compile as-is. Consider adding the appropriate MEAI provider integration package(s) to the list or adjusting the example to match the packages you recommend.

Suggested change



<PackageReference Include="Azure.AI.OpenAI" Version="2.*" />



<PackageReference Include="OpenAI" Version="2.*" />



<PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="9.*" />

<PackageReference Include="OpenAI" Version="2.*" />



<PackageReference Include="Microsoft.Extensions.AI.AzureOpenAI" Version="9.*" />

<PackageReference Include="Azure.AI.OpenAI" Version="2.*" />

Copilot · 2026-03-11T22:08:51Z

+   ```csharp
+   services.AddPredictionEnginePool<ModelInput, ModelOutput>()
+       .FromFile(modelPath);
+   ```


This skill recommends using services.AddPredictionEnginePool<,>() for ASP.NET Core, but the “Classic ML packages” list doesn’t include the package that provides PredictionEnginePool/AddPredictionEnginePool (commonly Microsoft.Extensions.ML). To make the guidance actionable, add that dependency to the recommended package list or clarify where it comes from.

Suggested change

```

Requires the Microsoft.Extensions.ML package, which provides PredictionEnginePool and AddPredictionEnginePool.

Copilot · 2026-03-11T22:08:51Z

+/plugins/dotnet/agents/optimizing-dotnet-performance.agent.md @dotnet/appmodel
+


This CODEOWNERS entry points to /plugins/dotnet/agents/optimizing-dotnet-performance.agent.md, but plugins/dotnet/ has no agents/ directory (the agent file exists under plugins/dotnet-diag/agents/). This rule looks like an accidental duplicate/typo and should be removed or corrected to the real path so ownership is applied as intended.

Suggested change

/plugins/dotnet/agents/optimizing-dotnet-performance.agent.md @dotnet/appmodel

moved from private to public fork

123971e

Copilot AI review requested due to automatic review settings March 4, 2026 00:51

jeffschwMSFT requested review from dbreshears and timheuer as code owners March 4, 2026 00:51

Copilot started reviewing on behalf of jeffschwMSFT March 4, 2026 00:52 View session

jeffschwMSFT requested review from artl93, danmoseley and luisquintanilla March 4, 2026 00:55

Copilot AI reviewed Mar 4, 2026

View reviewed changes

jeffschwMSFT added 4 commits March 5, 2026 13:18

add code owner

2e42175

accomodate feedback

c9595ab

update eval to make more natural language

ae80b4b

address eval feedback

9fad0f0

jeffschwMSFT requested a review from a team March 6, 2026 20:11

github-actions Bot mentioned this pull request Mar 7, 2026

🏥 Repository Health Dashboard #168

Closed

Merge branch 'main' into skill/dotnet-ai-ml

44e2632

danmoseley added the skill label Mar 9, 2026

ViktorHofer requested changes Mar 9, 2026

View reviewed changes

danmoseley requested a review from JeremyLikness March 10, 2026 00:18

artl93 approved these changes Mar 11, 2026

View reviewed changes

luisquintanilla approved these changes Mar 11, 2026

View reviewed changes

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

31d3efe

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 11, 2026 20:08

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

1f853bd

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

b2a3703

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Copilot started reviewing on behalf of jeffschwMSFT March 11, 2026 20:08 View session

jeffschwMSFT and others added 7 commits March 11, 2026 13:09

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

1679a64

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

24c9705

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

fde29ed

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

e622b0f

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

f10159b

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

7cd14fb

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Update plugins/dotnet-ai/skills/technology-selection/SKILL.md

a66e026

Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>

Copilot AI reviewed Mar 11, 2026

View reviewed changes

Comment thread .github/CODEOWNERS

Comment thread .github/CODEOWNERS Outdated

Comment thread plugins/dotnet-ai/skills/technology-selection/SKILL.md

Comment thread plugins/dotnet-ai/skills/technology-selection/SKILL.md

update supporting documentation of adding a new plugin

01aad99

ManishJayaswal approved these changes Mar 11, 2026

View reviewed changes

update claude-plugin

a05d1b5

Copilot AI review requested due to automatic review settings March 11, 2026 20:58

jeffschwMSFT enabled auto-merge (squash) March 11, 2026 20:59

Copilot AI reviewed Mar 11, 2026

View reviewed changes

Comment thread plugins/dotnet-ai/skills/technology-selection/SKILL.md

Comment thread plugins/dotnet-ai/skills/technology-selection/SKILL.md

dbreshears added the area-ai label Mar 11, 2026

jeffschwMSFT disabled auto-merge March 11, 2026 22:01

jeffschwMSFT and others added 2 commits March 11, 2026 15:04

Merge branch 'main' into skill/dotnet-ai-ml

d62e993

Copilot AI review requested due to automatic review settings March 11, 2026 22:05

Copilot started reviewing on behalf of jeffschwMSFT March 11, 2026 22:05 View session

jeffschwMSFT merged commit 551461a into dotnet:main Mar 11, 2026
17 of 18 checks passed

Copilot AI reviewed Mar 11, 2026

View reviewed changes

danmoseley mentioned this pull request Mar 14, 2026

dotnet-ai evaluation OOM-killed on standard runners #364

Open

jeffschwMSFT deleted the skill/dotnet-ai-ml branch March 16, 2026 23:13

	PackageReference Include="System.Numerics.Tensors" Version="10.*"
	<PackageReference Include="System.Numerics.Tensors" Version="10.*" />

		/plugins/dotnet/agents/optimizing-dotnet-performance.agent.md @dotnet/appmodel

Conversation

jeffschwMSFT commented Mar 4, 2026

Uh oh!

jeffschwMSFT commented Mar 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeffschwMSFT commented Mar 6, 2026

Uh oh!

jeffschwMSFT commented Mar 9, 2026

Uh oh!

ViktorHofer left a comment

Choose a reason for hiding this comment

Uh oh!

ManishJayaswal commented Mar 10, 2026

Uh oh!

artl93 left a comment

Choose a reason for hiding this comment

Uh oh!

jeffschwMSFT commented Mar 11, 2026

Uh oh!

github-actions Bot commented Mar 11, 2026

Skill Validation Results

Uh oh!

luisquintanilla left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects