harness.failharness.failharness.failharness.failharness.fail

One note per harness: what each score rests on, with the sources behind it.

Aider

.aiderignore, /read-only, --add-gitignore-files — long-standing, gitignore syntax (FAQ, options, commands) (A1/A3 ●, A5 ◐); the ignore file is one per repository, "default: .aiderignore in git root" (A2 ◐). Aider's workflow is explicit-add — files enter context when the user adds them — which gates context by construction but is a workflow property, not an enforcement claim (B1 ◐). Enforcement locus and shell coverage undocumented.

Claude Code

permissions.deny with Read()/Edit() path rules (permissions docs); project settings can be committed with the repo (.claude/settings.json), but Claude Code reads that one file "from the session's primary working directory" (settings), with no per-directory files (A2 ◐). Vendor states enforcement is deterministic: "Permission rules are enforced by Claude Code, not by the model" (D1 ●), with documented precedence "deny, then ask, then allow". A Read deny also blocks Edit and Write on the same path (B3 ●), evaluated before the tool call executes, and edits to permissions reach "the running session without a restart" (B4 ●) — but for context surfaces (Grep, Glob, @-mentions) the vendor's own wording is a "best-effort attempt", and NotebookEdit isn't covered (B1 ◐). Shell coverage is explicitly partial: recognized file commands only, rules "don't apply … to arbitrary subprocesses"; the opt-in OS sandbox closes that, with a default-enabled but prompt-gated unsandboxed-retry escape hatch (B2 ◐). Invalid managed sandbox values fail closed per the changelog (2.1.283), and an unparseable managed settings file stops every session; one the OS denies reading starts the session without its policies (2.1.285), and malformed mcp__ parameter rules are skipped with a warning (C2 ◐). Scoped rules "don't change which tools Claude sees. Claude Code checks them when Claude attempts a call, leaving the prefix intact", and a bare tool-name deny removes the tool rather than describing it (prompt caching) (C4 ●). CVE-2025-54794 was a prefix-matching-instead-of-canonicalization bug, fixed; the permissions docs now check both the requested path and the file a symlink resolves to, and symlink bypasses of Read deny rules through @-mentions and IDE selections were fixed in 2.1.289–2.1.290 (D2 ◐). Permission modes state that "Deny rules block in every mode, including bypassPermissions" — the blanket bypass lifts prompts, not deny rules (C1 ●) — and writes to protected paths, .claude settings among them, cannot be pre-approved by allow rules, yet in auto mode (the starting mode since 2.1.284) a model classifier decides them, and bypassPermissions allows them outright (C3 ◐). That protected-path restriction is the D3 gap: user deny rules lift only when the user edits them, but the built-in restriction falls to a blanket mode and, in auto mode, to a classifier model's judgment (D3 ◐). Subagent docs cover the delegation interplay partially — a subagent cannot self-declare bypassPermissions, and background subagents surface permission prompts to the main session — and state that built-in subagents inherit the parent conversation's permission rules, with no such statement for custom ones (D4 ◐).

Cline

.clineignore, gitignore syntax, one file "in your project root" (A1/A3 ●, A2 ◐, A5 ◐) — and the docs state plainly that it "is not a security or access-control boundary — ignored files can still be read via explicit @ mentions or shell commands": it controls automatic loading, not access (B1 ○, B2 ○). The page carries a deprecation banner and offers a PreToolUse hook script as the enforced replacement, which "actively blocks the tool call" when a read, edit, or shell command targets an ignored file: opt-in, user-installed code that does not resolve symlinks and is disabled in --yolo mode (D1 ◐). Writes and the rest of groups C–D are undocumented (?).

Cursor

.cursorignore, gitignore syntax, one file "in your root directory", with an opt-in setting that searches parent directories, not subdirectories (ignore docs) (A1/A3 ●, A2 ◐, A5 ◐). The docs disclaim completeness — "complete protection isn't guaranteed due to LLM unpredictability" — and state plainly that agent terminal and MCP tools "cannot block access" to ignored code (B1 ◐, B2 ○). Whether the rules gate writes, and when rule changes take effect, is undocumented (B3/B4 ?). CVE-2025-64110 was an ignore-file fail-open (a new .cursorignore invalidated existing protections), fixed in 2.0 (C2 ◐). CVE-2026-50548 and CVE-2026-50549 (Cato Networks) let the agent write files outside the workspace through a sandbox write-path override and a failed-canonicalization fallback through an in-workspace symlink; both fixed in Cursor 3.0 per the NVD records (D2 ◐). "Cursor blocks access to files listed in .cursorignore", under the same no-guarantee caveat (D1 ◐).

DeepSeek Harness

A named per-session abstraction — executors run "under the session file policy" (v0.1.6-alpha.1, carried into v0.1.7-rc.1) — documented as one of three sandbox modes (read-only, workspace-write, danger-full-access) over a workspace root, with no path patterns (sandbox docs) (A1/A2/A3 ○, B2/D4 ◐). An approved escalated retry "is a new call with a wider policy" (C1 ○), and danger-full-access, which "bypasses confinement", ships in the default preset table (permission presets) (D3 ○). Enforcement fixes track the classic classes: POSIX symlink/parent-directory combinations, Windows drive-relative paths (v0.1.6-alpha.1), cross-workspace deletion escapes (v0.1.7-alpha.1) (B3 ◐, D2 ◐). CVE-2026-82533 (OX Research): the sandboxed agent could reach the harness's own unauthenticated control interface and elevate its session to danger-full-access — the enforcement layer operable by the thing it governs; fixed August 27, 2026 in 0.1.2-alpha.1, whose release notes require a one-time token for network access to the web interface (C3 ◐: the bar exists now, its guarantee is undocumented). SAFETY.md states plainly that sandboxing, approval prompts, and permission controls "do not guarantee isolation or prevent damage" (D1 ◐).

Gemini CLI

.geminiignore plus .gitignore reuse (docs) (A1/A3 ●, A5 ◐); .geminiignore is one file "in the root of your project directory" — nested .gitignore files are honored per directory (source), but that is the version-control file, not the harness's policy file (A2 ◐). read_file enforces deterministically via shouldIgnoreFile() (B1/D1 ●) — and a feature request on the project's tracker names cat via run_shell_command as the workaround for reading ignored files (#13775); only the opt-in tool sandbox, off by default, turns ignored paths into kernel-denied paths for shell commands (source) (B2 ◐). Changes require a restart to take effect, per the same docs (B4 ◐). Negation handling has shipped a bug that errs restrictive (! rules fail to un-ignore, #5444, closed by the stale bot without a fix), but the error paths fail open: the matcher library skips a pattern ending in an unescaped backslash without a warning (node-ignore), an unreadable .geminiignore loads as no rules (source), and the ignore check returns not-ignored on any exception (source) (C2 ○). The write tools carry no ignore check at all — write_file and edit never consult the ignore service, verifiable in source; the opt-in sandbox denies only shell writes to ignored paths (B3 ◐). Nothing protects .geminiignore itself: "If the workspace is writable, we allow editing .gitignore and .geminiignore by default" (sandbox source) (C3 ○). read_file checks the ignore rules against both the requested path and its resolved real path (source); other surfaces are undocumented (D2 ◐).

GitHub Copilot (agent surfaces)

Content exclusion exists and is deterministic for completions, chat, and review — and the docs exclude both agent surfaces by name: "GitHub Copilot CLI and Agent mode in Copilot Chat in IDEs do not support content exclusion" (configuring exclusions). The path rules that do reach an agent surface are Copilot CLI's own: Read(...), Edit(...), and Write(...) glob rules in enterprise managed settings, session --deny-tool patterns such as read(.env) and write(PATH), and deniedPaths for sandboxed commands (configuration reference, command reference) (A1/A3 ●); agent mode in IDEs documents no equivalent. None of them lives in the tree: the repository settings file .github/copilot/settings.json supports only a listed set of keys, permission rules not among them, and content exclusion is server-delivered (A2 ○). Coverage is CLI-only: Read rules gate file reads, Edit/Write rules also cover a recognized set of shell redirections and in-place sed edits, but when the target "comes from a variable" the path rules "do not apply" (B1/B2/B3 ◐); rules are checked per request, and managed settings re-apply hourly "without restarting the session" (B4 ◐). A write(PATH) match "resolves symlinks and . / .. segments", while content exclusion documents symlinks and remote filesystems as not covered, with up to 30-minute propagation delay (D2 ◐). The CLI's rules are enforced by the harness, but the restriction an organization declares through content exclusion is not enforced on either agent surface (D1 ◐). File-based managed settings are rejected when they are "symlinks, not owned by root, or world-writable"; the docs state no such guard for the user-level settings file (C3 ◐). v1.0.88 closed a management-layer fail-open where ACP, AHP-host, and --server sessions "previously ran with no managed MCP, permission, or plugin policy". Copilot CLI's own deny rules hold against grants — "Deny rules always take precedence over allow rules, even when --allow-all is set or a matching approval has been saved" (allowing tools) — while agent mode in IDEs documents no equivalent (C1 ◐).

goose

.gooseignore is gone from the current tree: the docs carry no page for it, the string survives only in the repo's own .gitignore, and v1.44.0's "Remove stale gooseignore references" was the cleanup — current permissions are per-tool (allow / ask / deny), not per-path, so no declarative path-rule surface exists today (A1/A2/A3 ○). v1.49.0 shipped "Give permission denies precedence" (#11477), so a saved never_allow now beats an overlapping always_allow — but Auto mode approves every tool without consulting it (permission inspector) (C1 ◐), and hooks carry a documented deny contract (v1.44.0), though a policy hook that fails lets the call through unless set to on_failure: block (hooks docs) (D1 ◐). What the per-tool permissions gate operationally is undocumented (B1–B4 ?).

JetBrains AI Assistant / Junie

.aiignore, honoring .cursorignore/.codeiumignore/.aiexclude "as long as they are located in the root folder of the project" (A5 ◐), plus .noai "in the root directory of the project" as a whole-project kill switch — a vendor-specific marker, per-agent in the binary, single-vendor sense (A4 ◐) (docs); the documented locations are the project root, not per directory (A2 ◐). Enforcement is vendor-admitted best-effort: "ignored files may still be processed due to unforeseen issues" (docs), and file names and paths stay visible (support KB) (B1/D1 ◐). Allowlisted commands skip .aiignore checks entirely (B2 ○), and Brave Mode bypasses the approval prompts wholesale (D3 ○). Edits to ignored files are approval-gated rather than hard-blocked — Junie "will ask for explicit approval before viewing or editing the contents" (B3 ◐).

OpenAI Codex CLI

The long-standing read-restriction gap has begun closing: beta permission profiles add per-path filesystem rules — glob deny entries that deny "both reads and writes" — configurable in project .codex/config.toml files, which Codex loads "from the project root to your current working directory", closest wins, trusted projects only (advanced config): the files travel with the repo, but which ones apply depends on where the session starts, not on the path being accessed (A1/A2 ◐, A3 ◐, B1 ◐: beta, config-based, sandbox-enforced; the .codexignore request #1397 was closed into #2847, itself closed completed June 2026 with this feature as the answer). The profile semantics lean fail-closed: deny takes precedence over write and write over read, and "missing or empty filesystem tables keep filesystem access restricted" with a startup warning (C2 ◐); the :workspace profile keeps the workspace .codex directory read-only by default (C3 ◐); the deny list is enforced in sandbox code, but the harness also writes its paths and globs into model context as a developer message (source) (C4 ◐). The enforcement substrate is an always-on OS sandbox (macOS Seatbelt, Linux bubblewrap; docs, independently investigated) confining writes and network at kernel level, with spawned commands inheriting the same boundaries (B2/B3 ●, D1 ●, D4 ●); whether an edited profile reaches a running session without a restart is undocumented (B4 ◐). On Linux, literal deny paths are kept alongside their resolved targets so denials hold across writable symlinks (#48155) (D2 ◐). Approved escalations run unsandboxed and a danger-full-access mode exists (D3 ◐); approved commands "retain explicit filesystem denials" (rust-v0.159.0), but the :danger-full-access profile "removes local sandbox restrictions" (permissions) (C1 ◐).

Pi

Ships no file-access rules: an issue on its tracker lists the only mechanisms as the coarse --tools flag, custom TypeScript hooks, and interactive confirmation (#4459, which proposed native allow/deny policy files — auto-closed during a refactor without review) (A1–B4 ○, C1–C4 —). The security doc places safety outside the harness — it "comes from limiting the files, credentials, processes, and network services Pi can access" — and spells out the consequence: the working folder "does not prevent commands from accessing other paths available to the Pi process," project trust "does not limit what tool calls can access," and child processes "run with those same permissions" (D4 ○). What Pi does have is a deterministic enforcement point: the tool_call hook can block operations in host-side code before execution, and the shipped permission-gate example blocks by default when no UI is present — a policy layer could attach there, but enforcement itself is user-written code, not a shipped feature (D1 ◐).

Windsurf / Devin

.devinignore (with legacy .windsurfignore / .codeiumignore "still read and enforced alongside" it), gitignore syntax, one file at "the root of your repository": paths matched "are excluded from indexing and cannot be viewed, edited, or created by Devin" (docs) (A1/A3 ●, A2 ◐, A5 ◐, B1/B3 ●). Since v3.9.19 removed Cascade (changelog), Devin Local is the only agent in Devin Desktop, and it shares Devin CLI's permission rules, evaluated in a fixed order where "A deny rule always wins" (C1 ◐: only organization-level rules are guaranteed against Bypass mode). Smart mode's model judgment "only applies where no rule already decides the call, so a deny rule blocks the action"; the .devinignore docs do not name where it is enforced (D1 ●). An opt-in OS sandbox hides Read(...)-denied paths from shell commands (B2 ◐) and errs restrictive: unsupported exclusion rules are "ignored with a warning", and a sandbox that cannot be resolved refuses to start (C2 ◐). The write tools "refuse to write through a symlink" (v3.6.27), with reads unstated (D2 ◐), and background subagents get only tools already approved in the session (D4 ◐).

Zed

private_files and read_only_files glob settings, project-resident (configuration, tool permissions); project settings files can also sit in subdirectories (configuring Zed), and the agent's file tools look the settings up for the path being accessed (source) (A1/A2/A3 ●, B1/B3 ●). v1.21.0 added "..." inheritance-extension semantics for read_only_files, and v1.22.0 spread them to more glob settings. The terminal tool is governed by command-pattern rules, not path rules, and under the new OS sandbox terminal commands "can read most of the filesystem" (B2 ○). v1.22.0 stopped one invalid pattern from discarding the valid exclusion/read-only/private rules beside it, which narrows a fail-open in their own dialect rather than closing it: a matcher that cannot be built still matches nothing (#64525) (C2 ◐). always_deny rules are "highest priority, cannot be overridden" and still block when every tool action is auto-approved (C1 ●); agent edits inside .zed/, where project private_files live, always prompt (source), but sandboxed terminal commands can still write project files (C3 ◐). Enforcement is deterministic editor code, confirmed by the project's own advisory for CVE-2026-27967 — agent file tools checked private_files on logical paths, so symlinks escaped; patched 0.225.9 (D1 ●, D2 ◐).