Skip to content

Add first-class ARC/DinD compatibility mode for AWF#2839

Merged
lpcox merged 7 commits into
mainfrom
copilot/awf-fix-gaw-arc-runner-validation
May 10, 2026
Merged

Add first-class ARC/DinD compatibility mode for AWF#2839
lpcox merged 7 commits into
mainfrom
copilot/awf-fix-gaw-arc-runner-validation

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 10, 2026

  • Inspect the failing PR comment, recent workflow runs/logs, and the touched test files
  • Reproduce the failure locally with existing lint/build/test commands and targeted tests
  • Make the smallest fix for the failing test and validate the affected area
  • Scan changed files for secrets, run final validation, and reply to the PR comment

Copilot AI changed the title [WIP] Fix GAW missing first-class ARC runner validation for AWF workflows Add first-class ARC/DinD compatibility mode for AWF May 10, 2026
Copilot AI requested a review from lpcox May 10, 2026 14:23
Copilot finished work on behalf of lpcox May 10, 2026 14:23
@lpcox lpcox marked this pull request as ready for review May 10, 2026 15:23
@lpcox lpcox requested a review from Mossaka as a code owner May 10, 2026 15:23
Copilot AI review requested due to automatic review settings May 10, 2026 15:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 10, 2026

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 3845610

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 87.37% 87.40% 📈 +0.03%
Statements 87.34% 87.39% 📈 +0.05%
Functions 82.70% 82.46% 📉 -0.24%
Branches 79.43% 79.52% 📈 +0.09%
📁 Per-file Coverage Changes (6 files)
File Lines (Before → After) Statements (Before → After)
src/services/api-proxy-service.ts 98.5% → 98.5% (+0.04%) 98.5% → 98.5% (+0.06%)
src/services/agent-service.ts 97.5% → 97.6% (+0.11%) 97.6% → 97.8% (+0.16%)
src/services/agent-volumes.ts 91.6% → 91.7% (+0.16%) 91.8% → 92.0% (+0.22%)
src/cli.ts 28.3% → 28.6% (+0.30%) 28.3% → 28.6% (+0.30%)
src/services/cli-proxy-service.ts 94.7% → 95.2% (+0.50%) 94.7% → 95.5% (+0.72%)
src/container-lifecycle.ts 87.1% → 88.2% (+1.14%) 87.5% → 88.6% (+1.11%)
✨ New Files (1 files)
  • src/arc-dind.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a first-class ARC/DinD compatibility mode to AWF so it can run on split runner/Docker-daemon filesystems by rewriting AWF-managed bind-mount source paths under a staging prefix (/tmp/gh-aw/arc-root) and providing CLI/config/schema/docs support plus detection warnings.

Changes:

  • Introduces --arc-dind / container.arcDind and wires it through config types, config-file mapping, CLI option parsing, and JSON schemas.
  • Adds ARC/DinD bind-source translation (translateArcDindBindSource) and applies it to AWF-managed bind mounts across agent volumes and sidecars (Squid/API proxy/CLI proxy/iptables-init).
  • Documents ARC/DinD staging expectations and adds unit tests covering translation and compose output rewrites.
Show a summary per file
File Description
src/types/config.ts Adds WrapperConfig.arcDind with detailed behavior docs.
src/services/squid-service.ts Rewrites Squid bind-mount sources when arcDind is enabled.
src/services/cli-proxy-service.ts Rewrites CLI proxy log + CA cert bind-mount sources when arcDind is enabled.
src/services/api-proxy-service.ts Rewrites API proxy log bind-mount sources when arcDind is enabled.
src/services/agent-volumes.ts Applies translation to AWF-managed agent/chroot-related bind sources.
src/services/agent-volumes.test.ts Adds integration-style assertions that compose volumes are rewritten under the staging prefix.
src/services/agent-service.ts Updates iptables-init service to translate its init-signal bind source.
src/schema.test.ts Updates schema parity test fixture to include arcDind.
src/config-file.ts Adds container.arcDind and maps it into CLI option space.
src/config-file.test.ts Adds validation + mapping coverage for container.arcDind.
src/compose-generator.ts Passes config into buildIptablesInitService so it can apply translation.
src/cli.ts Adds --arc-dind flag and warns when DOCKER_HOST suggests ARC/DinD; plumbs arcDind into runtime config.
src/awf-config-schema.json Extends schema with container.arcDind.
src/arc-dind.ts New module: detection helper + bind-source translation logic and constants.
src/arc-dind.test.ts Unit tests for detection + translation.
scripts/generate-schema.mjs Ensures schema generation includes container.arcDind.
docs/environment.md Documents ARC/DinD behavior, reference config, and staging contract.
docs/awf-config.schema.json Updates published docs schema with container.arcDind.
docs/awf-config-spec.md Documents container.arcDind--arc-dind mapping.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/arc-dind.ts:74

  • translateArcDindBindSource() isn’t idempotent: if a caller passes a path that’s already under ARC_DIND_BIND_PREFIX, it will be re-prefixed (e.g. /tmp/gh-aw/arc-root/... becomes /tmp/gh-aw/arc-root/tmp/gh-aw/arc-root/...). This can happen if users choose a workDir (or other paths) already inside the staging prefix. Consider short-circuiting when sourcePath === ARC_DIND_BIND_PREFIX or sourcePath starts with ${ARC_DIND_BIND_PREFIX}/.
export function translateArcDindBindSource(sourcePath: string): string {
  if (/^[A-Za-z]:[\\/]/.test(sourcePath) || sourcePath.startsWith('\\\\')) {
    return sourcePath;
  }

  if (!path.posix.isAbsolute(sourcePath)) {
    return sourcePath;
  }

  if (sourcePath === '/') {
    return ARC_DIND_BIND_PREFIX;
  }

  if (ARC_DIND_PASSTHROUGH_PATHS.has(sourcePath)) {
    return sourcePath;
  }

  if (ARC_DIND_PASSTHROUGH_PREFIXES.some(prefix => sourcePath === prefix || sourcePath.startsWith(`${prefix}/`))) {
    return sourcePath;
  }

  return path.posix.join(ARC_DIND_BIND_PREFIX, sourcePath.slice(1));
}
  • Files reviewed: 19/19 changed files
  • Comments generated: 3

Comment thread src/arc-dind.ts Outdated
Comment on lines +33 to +49
if (dockerHost.startsWith('unix://')) {
if (DEFAULT_DOCKER_HOST_SOCKETS.has(dockerHost)) {
return { detected: false };
}

return {
detected: true,
dockerHost,
reason: 'non-default-unix-socket',
};
}

return {
detected: true,
dockerHost,
reason: 'tcp',
};
Comment on lines 98 to 103
const copilotHomeDir = path.join(effectiveHome, '.copilot');
if (fs.existsSync(copilotHomeDir)) {
try {
fs.accessSync(copilotHomeDir, fs.constants.R_OK | fs.constants.W_OK);
agentVolumes.push(`${copilotHomeDir}:/host${effectiveHome}/.copilot:rw`);
agentVolumes.push(`${sourcePath(copilotHomeDir)}:/host${effectiveHome}/.copilot:rw`);
} catch (error) {
Comment on lines 57 to 62
volumes: [
// Log directory for HTTP server logs
`${cliProxyLogsPath}:/var/log/cli-proxy:rw`,
`${sourcePath(cliProxyLogsPath)}:/var/log/cli-proxy:rw`,
// Mount host CA cert for TLS verification
...(config.difcProxyCaCert ? [`${config.difcProxyCaCert}:/tmp/proxy-tls/ca.crt:ro`] : []),
...(config.difcProxyCaCert ? [`${sourcePath(config.difcProxyCaCert)}:/tmp/proxy-tls/ca.crt:ro`] : []),
],
@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented May 10, 2026

@copilot address the review feedback

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 10, 2026

@copilot address the review feedback

Addressed in 59b4819.

This follow-up narrows ARC/DinD auto-detection to explicit tcp:// hosts, makes the optional ~/.copilot bind check use the translated ARC/DinD source path, and adds focused regression tests for the translated .copilot gating and cli-proxy volume rewrites.

lpcox added a commit that referenced this pull request May 10, 2026
/dev and /sys are kernel virtual filesystems provided by the Docker
daemon itself, not runner files staged under a prefix path. Prefixing
them would look for non-existent directories under the runner root.

This aligns with the approach in PR #2839 which uses explicit
passthrough prefixes for these paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lpcox added a commit that referenced this pull request May 10, 2026
…2843)

* Initial plan

* feat: add docker host path prefix remapping

* fix: address docker host prefix review feedback

* fix: exclude /dev and /sys from docker host path prefix translation

/dev and /sys are kernel virtual filesystems provided by the Docker
daemon itself, not runner files staged under a prefix path. Prefixing
them would look for non-existent directories under the runner root.

This aligns with the approach in PR #2839 which uses explicit
passthrough prefixes for these paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add hidepid=2 to procfs mount and /proc to passthrough list

Defense-in-depth against credential leakage via /proc/*/environ:

1. entrypoint.sh: Mount procfs with hidepid=2 so awfuser cannot read
   PID 1's /proc/1/environ, which may briefly hold auth tokens before
   unset_sensitive_tokens() clears them. This also blocks indirect
   access via /dev/fd → /proc/self/fd symlink traversal.

2. agent-volumes.ts: Add /proc to the kernel virtual filesystem
   passthrough list alongside /dev and /sys. While /proc is not
   bind-mounted (it uses mount -t proc), this is a defensive guard
   against accidental exposure through path prefix translation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The --docker-host-path-prefix flag from PR #2843 fully supersedes the
--arc-dind feature. This merge brings in main (which includes #2841 and
#2843) and removes the now-redundant arc-dind module and inline
sourcePath() wrappers.

Changes:
- Delete src/arc-dind.ts and src/arc-dind.test.ts (orphaned)
- Remove --arc-dind CLI option, config, schema, and docs
- Remove sourcePath() wrappers from all service builders
- Keep --docker-host-path-prefix with centralized translation in
  buildAgentVolumes() and kernel VFS passthrough (/dev, /sys, /proc)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented May 10, 2026

@copilot fix this failing test #2839 (comment)

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 10, 2026

@copilot fix this failing test #2839 (comment)

Addressed in 8be1476. The build-test workflow now uses plain git clone https://github.com/... for the public test repositories instead of gh repo clone, so it no longer depends on GH_TOKEN for those clones.

Copilot finished work on behalf of lpcox May 10, 2026 19:30
Copilot AI requested a review from lpcox May 10, 2026 19:30
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Smoke Test: Copilot BYOK (Offline) Mode

Test Result
GitHub MCP connectivity ⚠️ 401 (MCP not authenticated in sandbox)
GitHub.com HTTP ✅ (pre-step verified)
File write/read (smoke-test-copilot-byok-25637703141.txt) ✅ Content confirmed
BYOK inference (agent → api-proxy → api.githubcopilot.com) ✅ Responding normally

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: PASS

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

  • ❌ GitHub MCP Testing (gh pr list: auth issue)
  • ✅ Playwright Testing (GitHub page loaded)
  • ✅ File Writing Testing (file created)
  • ✅ Bash Tool Testing (file verified)

Status: PARTIAL (3/4 passed; gh auth config issue)

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

  • 🔌 GitHub MCP: ❌ (401 Bad credentials — token not propagated)
  • 🌐 GitHub.com HTTP: ⚠️ (pre-step template vars not expanded)
  • 📁 File write/read: ✅ (smoke-test-copilot-25637703145.txt confirmed)

Overall: FAIL — MCP connectivity could not be verified.

Pre-step file: "Smoke test passed for Copilot at Sun May 10 19:40:00 UTC 2026"

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Codex: FAIL
PR: Fix chroot bind mounts for ARC/DinD split runner-daemon filesystems
PR: Honor Unix DOCKER_HOST sockets for DinD mounts on ARC runners
GitHub PR review ✅ / SafeInputs ❌ / Playwright ✅ / Tavily ❌
File+Bash ✅ / Discussion ❌ / Build ✅
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results — FAIL

Check Result
Redis PING ❌ No response (timeout)
PostgreSQL pg_isready ❌ No response
PostgreSQL SELECT 1 ❌ Not attempted (pg_isready failed)

host.docker.internal is unreachable on both ports 6379 and 5432. Service containers may not be running in this environment.

🔌 Service connectivity validated by Smoke Services

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color passed ✅ PASS
Go env passed ✅ PASS
Go uuid passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Note (Java): The default ~/.m2/repository was owned by root and not writable. Maven was run with -Dmaven.repo.local=/tmp/gh-aw/agent/m2-repo as a workaround; all tests passed successfully.

Generated by Build Test Suite for issue #2839 · ● 665K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[awf] compiler: GAW missing first-class ARC runner validation and compatibility for AWF workflows

3 participants