Skip to content

Add build perf baseline eval tests#128

Merged
JanKrivanek merged 3 commits into
mainfrom
dev/jankrivanek/bld-perf-baseline-skilleval
Feb 26, 2026
Merged

Add build perf baseline eval tests#128
JanKrivanek merged 3 commits into
mainfrom
dev/jankrivanek/bld-perf-baseline-skilleval

Conversation

@JanKrivanek
Copy link
Copy Markdown
Member

No description provided.

Copilot AI review requested due to automatic review settings February 26, 2026 06:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds evaluation test scenarios for the build-perf-baseline skill, which helps developers establish build performance baselines and identify optimization opportunities. The PR creates a multi-project .NET solution intentionally designed with build performance issues (redundant project references, unnecessary analyzer settings) for the AI agent to detect and diagnose.

Changes:

  • Added a new test scenario eval.yaml that instructs the AI to measure cold/warm/no-op builds and identify optimization opportunities
  • Created a 5-project .NET solution (Common → DataAccess → BusinessLogic → WebApi + Tests) with intentional performance anti-patterns
  • Configured Directory.Build.props with performance-impacting settings like GenerateDocumentationFile and EnforceCodeStyleInBuild always enabled

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
eval.yaml Test scenario definition with assertions for build baseline measurement and optimization recommendations
PerfBaseline.sln VS solution file defining 5 projects with multiple configurations
PerfBaseline.slnx Empty XML-based solution file (issue: should contain project entries or be removed)
Directory.Build.props Shared build properties with intentionally slow settings (doc generation, analyzers always on)
Common/Common.csproj Base library project with entity models and utilities
Common/Models.cs Entity, Result, and StringExtensions implementation
DataAccess/DataAccess.csproj Data access layer depending on Common
DataAccess/Repository.cs Simple in-memory repository implementation
BusinessLogic/BusinessLogic.csproj Business logic layer with intentional redundant Common reference
BusinessLogic/EntityService.cs Service layer wrapping repository operations
WebApi/WebApi.csproj API entry point with intentional redundant transitive references
WebApi/Program.cs Console application demonstrating service usage
Tests/Tests.csproj Test project with intentional redundant references (issue: missing OutputType)
Tests/EntityServiceTests.cs Simple test implementations with Main method

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/dotnet-msbuild/tests/build-perf-baseline/PerfBaseline.slnx Outdated
Comment thread src/dotnet-msbuild/tests/build-perf-baseline/Tests/Tests.csproj
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 26, 2026

Skill Validation Results

Skill Scenario Baseline With Skill Δ Skills Loaded Verdict
build-perf-baseline Establish build performance baseline and recommend optimizations 3.3/5 4.0/5 +0.7 ✅ build-perf-baseline; binlog-generation; tools: skill, binlog-mcp-load_binlog, binlog-mcp-list_projects, binlog-mcp-get_expensive_tasks, binlog-mcp-get_expensive_targets, binlog-mcp-get_expensive_projects, binlog-mcp-get_node_timeline, binlog-mcp-search_targets_by_name, binlog-mcp-search_tasks_by_name, binlog-mcp-get_expensive_analyzers

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

@JanKrivanek JanKrivanek changed the title [WIP] Add build perf baseline eval tests Add build perf baseline eval tests Feb 26, 2026
@JanKrivanek JanKrivanek merged commit b0f531b into main Feb 26, 2026
7 checks passed
@JanKrivanek JanKrivanek deleted the dev/jankrivanek/bld-perf-baseline-skilleval branch February 26, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants