Silver Bullet Gaps Report
Evidence-only audit of 50 known AI SE & DevOps failure modes

SB requires UAT.md, REVIEW.md, SECURITY.md to exist as files, but it does not run them. The hook never executes the tests or reviews. A confidently-wrong agent that wrote beautiful-looking artifacts containing fabricated "PASS" results would pass the artifact-existence gate. The UAT.md check in uat-gate.sh (lines 73-78) does grep for FAIL rows but trusts the agent's self-reported table.

SB matches the *first line* of a shell command and only enforces a blocklist of two hardcoded forbidden skills. There is no inspection of command arguments (e.g. blocking `rm -rf /`, `curl | sh`, `chmod 777`); no static analysis of edit operations; no `dev-cycle-check` guard against specific dangerous patterns. The hook fires on the *type* of command (commit, push, deploy), not its substance.

SB re-injects core rules on SessionStart, which is a static approach. There is no per-compaction re-injection; the hook only runs at startup, clear, resume, compact but only injects context at startup/clear. Drift between plan and execution is only caught at delivery via artifact existence checks, not continuously.

SB requires verify-tests to have run for plan seals, but it does not require code quality checks (lint, typecheck, complexity). The skill `silver-verify` produces UAT.md but does not enforce code-quality metrics.

There is a trivial-bypass file mechanism AND a user "SB OVERRIDE: reason" mechanism. The override is logged to `orchestrator.json` for audit, but trivial-bypass is silently honored — agents can create the trivial file (though trivial-file-guard.sh blocks Bash from creating it). Multiple backdoors exist: ci-red-override file, dev-prod drift, branch finishing exemption on main.

Approval is implicit in skill recording (silver-review, silver-review-request, silver-review-triage). There is no enforced external human signoff — the agent invokes the skills and they're "approved". No GitHub PR review approval requirement is enforced.

stop-check.sh can be bypassed by either: (a) writing fake state entries (the file is append-only text, not validated against action history), (b) creating fake artifacts (the file existence check does not inspect content depth). The artifact-existence check is a falsification-resistant gate only if the file is the real output of running real work — but nothing in the hook validates that the artifact's mtime/post-state is consistent with actual code changes.

record-skill.sh records skill names on EVERY Skill PostToolUse event, even if the skill was just `cat`'d or read. The skill-only metadata (read SKILL.md) is recorded into a separate `state.loaded` file, but a `Skill` invocation that produces no work still records. The artifact-existence check is the only falsification guard.

SB has the Context Mode integration (blocks edits without graphify, denies large reads). It re-injects core-rules at session-start. However, there is no per-compaction re-injection: the session-start hook only fires on `startup`/`clear` — `compact` events fall through without re-injecting (per v0.30.0 fix #87). So a long session that hits a compaction event mid-work loses the rules injection until next startup.

SessionStart handler reads `source` field and only re-injects core-rules on `startup`/`clear`. The `compact` and `resume` events fall through without re-injection — this is intentional (v0.30.0 fix #87). The downside: mid-session compact does NOT re-inject rules; the rules survive the compact only if the model had them in context already.

Same as #23: compact events are detected but not used to re-inject rules. SB has the state-file-on-disk (preserved across compactions) and session-ledger (preserved mid-session) but no active re-injection at compaction.

Previous report claimed 'Partial'. On inspection, instruction-file-guard.sh only blocks CREATION of root CLAUDE.md/AGENTS.md during init — it does not inspect CONTENT of instruction files for prompt injection. orchestrator-directive-guard.sh enforces that the directive queue is followed, but does not scan for injection patterns in the user prompt. There is no content inspection of .planning/, .cursor/, or external files for prompt-injection markers.

Parent-orchestrator mode has a tool allowlist. But for the normal session, the dev-cycle-check only blocks *source edits* under certain conditions; it does not constrain the agent's tool selection broadly. The agent still has the full Edit/Write/Bash tool surface — SB enforces "when can you edit" not "what tools can you call".

SB hooks fire on command *type* (commit/push/deploy) but do not block dangerous *arguments* like `rm -rf /`, `curl | sh`, shell expansion, command substitution, etc. The hooks can be bypassed by simply not using those exact verbs. There is no shell-arg sanitizer, no shell-quoting enforcement, no `set -o noexec` style guard.

Previous report claimed 'Partial'. Inspection confirms: session-log-init.sh writes a session log file with no redaction. The log includes '## Files changed' and '## Skills invoked' sections that the documentation step fills in — but no SB enforcement to scrub API keys, tokens, or PII from those files. There is no PreToolUse/Bash hook to block reading/writing .env, ~/.ssh/, or AWS credentials.

silver-secure skill names supply-chain as a check area, but there is no automated SB hook that runs `npm audit` / `pip-audit` / `trivy` before delivery. The skill is advisory; the completion-audit gate only checks that SECURITY.md exists, not that it actually contains a passing supply-chain scan.

SB has hook-level matchers, but does not configure MCP tool permissions or per-tool ACL. There is no SB enforcement of what MCP servers the host runtime has registered; the plugin-cache-guard prevents writes to SB's own plugin cache, but other MCP tools are not constrained.

Previous report claimed 'Solved'. On inspection, there is no enforcement against the agent *revealing* the system prompt or SB directives to the user. The override mechanism is logged, but if the user simply asks the model to dump the rules, nothing prevents that. orchestrator-directive-guard.sh enforces the *flow* of skills, not confidentiality of the prompt.

No SB hook blocks reads of ~/.ssh/, ~/.aws/, or .env files. The completion-audit gate does not scan for credentials. SB config has a `_notifications_comment` warning about webhook URLs but no enforcement.

SB's deployment gate ensures required_deploy skills are recorded, but does not check whether the deployment is happening from an approved CI environment vs a laptop. There is no SB hook to detect local prod-deploy attempts.

SB has RTK + semantic-compress + stack-optimizer (per .silver-bullet.json) but no per-prompt token-cost visibility. The session log has a 'Virtual cost' field that is filled at documentation step but never computed or tracked.

context-mode-read-deny + semantic-compress + graphify reduce bloat, but no per-prompt token counter or budget enforcement. RTK meta-commands (`rtk gain`) exist but are not enforced.

ordering check exists for review triad but only checks ordering, not whether prior decisions were referenced. session-log-init has '## Autonomous decisions' section but no enforcement that the agent actually reads prior decisions before acting.

doc-scheme-gate enforces that docs/CHANGELOG.md, docs/knowledge/, docs/learnings/ are updated on delivery. But there is no check that doc-content is actually current (mtime > last code change). Drift between code and docs is detected only at delivery.

silver-blast-radius and devops-quality-gates are skills (advisor prompts) that require the agent to think about IaC drift. There is no SB hook that runs `terraform plan` or `kubectl diff` and records the output as a pre-deploy gate.

Previous report claimed 'Solved'. Inspection: timeout-check.sh caps autonomous sessions at 10 minutes (Tier 1) and 100 tool calls (Tier 2). However, there is NO SB enforcement of per-API rate limits (e.g. blocking `gh api` calls that would exceed GitHub rate limits; blocking bulk `curl` that exceeds bandwidth). The "DoS" here is the agent itself burning through its own context, not a network DoS.

SB requires REVIEW.md to exist and that review-request/review/review-triage were called in the right order. But there is no depth check — REVIEW.md could be a one-liner saying "LGTM". The artifact-existence gate is satisfied by a 1KB placeholder.