Capstone C2 — Build a Smart Contract or Cloud Security Harness

Course: 2A — Building AI Harnesses for Cybersecurity Module: C2 — Build a Smart Contract or Cloud Security Harness Duration: 90 minutes Level: Senior Engineer and above Prerequisites: C1 complete. You have the harness architecture — scope middleware, memory, tools, evidence, triage, report — and you have benchmarked it. This capstone applies that architecture to a specialized domain and produces a publishable portfolio artifact.

This is a capstone. The teaching is design guidance for applying the C1 skeleton to smart contract or cloud security. The lab (artifact 07) is the build itself.

Learning Objectives

  1. Choose a domain (EVMbench-aligned smart contract audit or cloud posture + red team) and produce an authorization and scope document.
  2. Define the three modes the harness supports, the tool suite, the evidence schema, and the report format.
  3. Implement the full pipeline for the chosen domain and run it against a real lab target.
  4. Capture structured findings with full evidence records and generate a client-ready report.
  5. Score the harness against the domain benchmark and produce a one-page results summary for LinkedIn, GitHub, and Deepthreat.ai.

C2.1 — Design and Scoping

Fifteen minutes. You already have the architecture from C1. The design work here is domain-specific: choosing the track, defining the three modes, selecting the tools, and writing the authorization document.

Choose your domain

Two tracks:

Choose the track where you can stand up a real lab target in the build phase. For smart contracts, forked mainnet via Foundry or a DVD challenge is fast. For cloud, an isolated AWS account with Terraform-provisioned misconfigurations takes more setup but is the real thing.

Selecting your EVMbench subset (smart contract track)

EVMbench ships 117 curated vulnerabilities across 40 real EVM repositories. For a capstone you do not run all 117 — you select a representative subset of 3 to 5 and score against those. The selection discipline is what makes the subset meaningful rather than cherry-picked: span the vulnerability classes, not the easy wins. Pick one from each of these classes so the subset exercises different detection and exploitation paths:

  1. Reentrancy — the canonical class. Slither detects the checks-effects-interactions violation; the exploit must prove the reentrant call drains funds.
  2. Access control / privilege escalation — a public function that should be onlyRole. Tests whether the harness reasons about authorization intent, not just visibility. The PoC is the unauthorized call itself.
  3. Price oracle manipulation (DeFi-specific) — the class where purpose-built agents score 92% and general LLMs score 34% (S11.1, S12.2). Requires forked mainnet because the exploit depends on real protocol state (reserves, TWAP windows). This is the discriminative class.
  4. Arithmetic / invariant violation — an integer issue or logic bug that breaks a protocol invariant. Tests the invariant-extraction harness from S11.2.

Three to five spanning these classes runs in the build window but exercises both static-detectable patterns (reentrancy, access control) and semantic-reasoning patterns (oracle, invariant). Document which vulnerabilities you selected and why — the selection rationale is part of the benchmark's defensibility. A reviewer who sees "we picked these four because they span the four major classes" trusts the subset; one who sees four reentrancy bugs does not.

Define the three modes

The three modes are the spine of the harness. Each mode has its own tool chain and evidence shape:

Smart contract (Detect / Patch / Exploit):

Cloud (Detect / Remediate / Exploit):

A harness that only runs Detect is half a harness. The three modes together are what makes the benchmark meaningful — recall alone hides weaknesses in remediation and exploitation. This is the same argument from S11.1 and S12.1: detection without exploitation or patching is a hypothesis, not a delivered result.

Define tool suite, evidence schema, and report format

The tool suite is domain-specific, but the pattern from C1 holds: each tool has a Pydantic input model, a scope check, a high-impact flag, and a run method. For smart contracts the suite is Slither, Mythril, a Foundry test runner, a forked-mainnet PoC builder, and an evidence recorder; for cloud it is a configuration scanner (Prowler/AWS Config), an IAM analyzer, an attack-surface mapper, a remediation applier, and an evidence recorder.

Each suite maps to its three modes. Smart-contract Detect uses Slither (reentrancy, integer issues, access control), Mythril (symbolic execution for deeper paths), and LLM reasoning over reorganized source (the context engineering from S11); Patch uses the Foundry runner; Exploit uses the forked-mainnet builder. Cloud Detect uses Prowler (CIS scanning) and the IAM analyzer (over-permissive policies, escalation paths); Remediate uses the applier (IAM policy fixes, security-group changes, resource removal); Exploit uses the attack-surface mapper to find a reachable exposure and attempts the red-team action. The evidence recorder captures each stage.

The evidence schema is the same tamper-evident chain from C1: finding links to evidence (with sha256 hash) links to tool call links to log line (trace ID). The report format is domain-specific — the smart contract report follows the S12 six-section structure (scope, methodology, findings table, severity, detailed findings, remediation roadmap); the cloud report follows a posture structure (scope, control framework, findings by category, evidence, remediation roadmap).

Cascade verification (smart contract Patch mode)

The Patch mode is not "generate a diff and check it compiles." It is a cascade of verification gates from S11.4, each of which must pass before the patch is accepted:

  1. Detection clean. Re-run Slither and Mythril against the patched source. The original finding must be gone — and no new findings may appear. A patch that removes the reentrancy but introduces an integer overflow fails.
  2. Test suite passes. Run the Foundry test suite against the patched contract. Every existing test must pass — the patch preserved behavior. This is why you need a real test suite on the target; DVD and EVMbench repositories include them.
  3. Exploit fails. Re-run the Exploit-mode PoC against the patched contract. This catches the trap from S11.4: a patch that fixes the specific PoC transaction but leaves the underlying flaw (e.g., blacklisting one attacker address while leaving the access-control gap open). If a second PoC can still exploit the patched code, the patch is incomplete.

All three pass, or the patch is rejected and the LLM retries within a budget. The cascade is what makes Patch quality a meaningful score rather than a formality.

Write the authorization and scope document

For smart contracts, the authorization document defines: the contracts in scope (addresses and commit hashes), the forked-block number, what the harness is authorized to do (read, mutate in the fork, deploy PoCs), and the engagement boundary. For cloud, it defines: the AWS account ID, the regions in scope, the resources the harness may read and mutate, the actions it may take in Exploit mode (and the blast-radius limits), and the teardown procedure.

The authorization document is not a formality. For cloud especially, an exploit action against the wrong account or without blast-radius limits is a real incident. Write it before you build.

C2.2 — Build and Run

Sixty minutes. Implement the full pipeline and run it against a real lab target.

Implement the full pipeline for your chosen domain

The pipeline is the C1 harness with domain-specific tools and modes. The build order is the same: scope middleware, memory, tools, evidence, triage, report. The difference is the tools and the mode loop.

For smart contracts, the pipeline is:

  1. Load the target contract source and the EVMbench metadata (expected vulnerabilities).
  2. Detect mode — run Slither, Mythril, and the LLM reasoning over reorganized source. Collect candidate findings.
  3. Patch mode — for each detected finding, generate a patch. Verify by re-running detection (finding gone) and the Foundry test suite (behavior preserved).
  4. Exploit mode — for each detected finding, build a PoC on forked mainnet. Run it. Record success/failure.
  5. Triage — filter false positives, confirm findings with evidence.
  6. Report — generate the audit report.

Foundry forked-mainnet setup (smart contract track)

Exploit mode and the third cascade gate both require a forked mainnet. Foundry does this natively:

forge test --fork-url $ARCHIVE_RPC_URL --fork-block-number $BLOCK -vvv

Two things must be pinned and recorded in the evidence: the --fork-url (an archive-capable RPC — Alchemy/Infura paid tier or a self-hosted archive node) and the --fork-block-number. Pinning the block is what makes the PoC reproducible — the forked state at block N is deterministic, so a reviewer re-running with the same RPC and block gets the same contract storage, oracle prices, and liquidity. A PoC that worked on "whatever block is current" is not reproducible and therefore not trustworthy as evidence. This is also why oracle-manipulation exploits belong in the subset: they require real protocol state (reserves, TWAP windows) that only exists on a fork — a toy contract cannot reproduce them.

For cloud, the pipeline is:

  1. Load the target environment (account, regions) and the control framework (CIS subset).
  2. Detect mode — run the configuration scanner and IAM analyzer. Collect posture violations.
  3. Remediate mode — for each finding, generate and apply a remediation. Verify by re-running detection.
  4. Exploit mode — for each exposure, attempt a controlled red-team action proving it is real.
  5. Triage and report.

The dual pipeline: posture and red team together (cloud track)

The cloud track's distinguishing decision is that Detect and Exploit are not independent — they are a dual pipeline where posture findings feed red team hypotheses. This integrates S09 (posture) and S10 (red team) into one harness, and it is what separates a cloud harness from a CSPM tool.

The posture scanner (Prowler, AWS Config) runs Detect and produces control violations — an over-permissive IAM role, a public S3 bucket, a security group open to 0.0.0.0/0. Each posture finding is a hypothesis that the exposure is real and reachable. The red team harness (S10's Pacu-based engine) validates each hypothesis: can an untrusted principal actually assume that role? Can an anonymous caller read that bucket?

When the red team succeeds, the finding is confirmed exploitable and its severity is elevated — a "public S3 bucket" is Medium from a configuration standpoint but Critical when the red team proves it holds sensitive data. When the red team fails (the bucket is public but empty, or the role exists but no untrusted principal can reach the assumption path), severity is reduced or the finding is marked informational. The dual pipeline turns static configuration data into validated risk — the S09-to-S10 bridge, and the cloud track's core value proposition.

IAM graph construction from real AWS data

The IAM analyzer in Detect builds a privilege-escalation graph from live AWS data — the technique from S10.2. Enumerate principals (users, roles) and policies (managed and inline) via API calls (list-roles, list-attached-role-policies, get-policy-version, list-role-policies), then build a directed graph whose edges are the escalation primitives — sts:AssumeRole, iam:PassRole, iam:CreatePolicy, iam:AttachRolePolicy. A path from a starting principal to a privileged target is an escalation chain.

Build from live API calls, not Terraform state — live state is the truth, and drift between IaC and reality is itself a finding. The graph feeds Detect (the path is a posture violation) and Exploit (the red team traverses it). The evidence record captures the full chain: principal A assumes role B, B has iam:PassRole and passes role C to a service, C has admin — traceable link by link.

Run against a real lab target

This is non-negotiable. A harness tested only on toy examples is a hypothesis. The lab target makes the benchmark falsifiable — anyone can run the same target and compare their scores. This is the repeatable-lab discipline from S05.3: isolated targets, seeded vulnerabilities, consistent scoring methodology, fixed compute envelope.

Smart contract lab target options:

Cloud lab target options:

Run the full pipeline. Record every finding with its evidence. The success criterion is not "it ran" — it is "it produced confirmed findings with evidence for the known vulnerabilities/misconfigurations." A run that produces zero confirmed findings against a target with known bugs is a failed run. Debug the pipeline: did Detect find the bugs? Did the triage filter reject them erroneously? Did the evidence recorder fail to capture proof? Each failure point is a diagnostic.

Capture structured findings with full evidence records

Every confirmed finding carries the full evidence chain from C1: finding ID, mode, tool call, raw output (PoC transaction, remediation diff, scanner output), sha256 hash, trace ID. A finding without evidence is an opinion; a finding with a tamper-evident chain is a result.

For smart contracts, an Exploit-mode evidence record contains the PoC transaction hash, the forked-block number, the state diff (before and after), and the Solidity code of the exploit contract — a reviewer can replay the PoC on the same forked block and verify the same state change. For cloud, it contains the API call chain (e.g., AssumeRole → ListObjects → GetObject), the credentials used (scoped to the isolated account), and the data accessed — a reviewer can verify the blast radius was contained.

The triage filter from C1 applies unchanged: candidate findings move to confirmed only with linked evidence and a passing confidence threshold; known false positives are filtered by rule (code, not LLM judgment); duplicates are merged. The state machine is domain-independent — what changes is the false-positive rule set (e.g., for smart contracts, a Slither finding on a test file is a false positive; for cloud, a Prowler finding on a resource in an excluded region is a false positive).

Generate a client-ready report

Run the report generator on the confirmed findings. The report must be deliverable as-is:

The report is dual-output: JSON for machine consumption and HTML for the human reviewer.

C2.3 — Benchmark and Publish

Fifteen minutes. This is the portfolio moment. The benchmark score and the client report distill into a single publishable page.

Score against the domain benchmark

Smart contract (EVMbench subset): Score the three modes independently:

Report all three together. A harness scoring 85% Detect / 30% Patch / 30% Exploit is a detection engine, not an audit harness. The three scores together are the benchmark. The Patch quality gate has two parts: re-run detection on the patched code (the finding must be gone, and no new findings introduced), and run the Foundry test suite (all tests must pass — behavior preserved). A patch that removes the bug but breaks a test fails. A patch that fixes the specific PoC but leaves the underlying flaw fails (the third cascade gate catches this). Both gates must pass.

Cloud (posture benchmark): Score the three modes:

The Remediation success gate also has two parts: re-scan the resource after applying the fix (the control must now pass), and verify the application still functions (the remediation did not break it — e.g., tightening an IAM policy did not revoke a permission the app needs). Over-remediation — removing all permissions to make the scan clean — breaks the application and fails the gate. The correct remediation tightens to least-privilege, not to zero-privilege.

A worked example: scoring a smart contract run

Suppose your EVMbench subset is four vulnerabilities: a reentrancy bug, an access-control gap, an oracle-manipulation path, and an integer overflow. The harness produces:

# Vulnerability Detect Patch Exploit
1 Reentrancy Found Passed cascade (detection clean, tests pass, exploit now fails) PoC drained funds on fork
2 Access control Found Passed cascade PoC executed unauthorized call
3 Oracle manipulation Found Patch broke a test (retry budget exhausted) PoC failed — flash-loan path not constructed
4 Integer overflow Missed — —

Scoring: Detect recall = 3 of 4 = 75%. Patch quality = 2 of 3 detected = 67%. Exploit success = 2 of 3 detected = 67%.

What the numbers tell you: detection missed the integer overflow (a Slither-detectable class — check the Slither wrapper or the confidence threshold). Patch failed on the oracle case because the patch broke a test. Exploit failed on the same case because flash-loan construction is the hardest PoC (it requires understanding reserves, routing, and timing — exactly where general LLMs score 34% per EVMbench). Each number points at a specific subsystem to improve. A single "detection rate" would hide all three failure modes. The cloud track's three scores follow the same diagnostic logic — a posture miss points at the scanner, a failed remediation points at the least-privilege logic, a failed exploit (e.g., an open port with no live service) points at the red-team's reachability check.

Produce a one-page results summary

Distill the benchmark into a single page. This page has four required components:

1. The benchmark table. The three-mode scores, with the target and subset explicit:

[Harness Name] — Smart Contract Security Harness Benchmark

Target: EVMbench subset (4 vulnerabilities: reentrancy, access control,
        oracle manipulation, integer overflow) on forked mainnet
Forked block: 19,200,000
Date: 2026-07-09

| Mode        | Score | Detail                          |
|-------------|-------|---------------------------------|
| Detect      | 75%   | 3 of 4 vulnerabilities found    |
| Patch       | 67%   | 2 of 3 detected patched cleanly |
| Exploit     | 67%   | 2 of 3 detected exploited on fork |

Client report: [link to HTML report]
Evidence chain: [link to repo with evidence records + fork config]

2. A methodology note. Two to three sentences stating what was tested and how: the tools, the cascade gates applied, the forked-block pinning for reproducibility, and the compute envelope (model, approximate token cost — the S05.3 fixed-compute discipline). The methodology is what lets a reviewer judge whether the number is meaningful; without it the score is an assertion.

3. An architecture diagram reference. A link to or thumbnail of the three-mode pipeline (Detect → Triage → Patch/Exploit → Evidence → Report). It shows a reviewer that this is a system, not a script. Reference the S11 architecture if your diagram follows it.

4. Honest limitations. State what the benchmark does not claim: the subset size (4 vulnerabilities, not 117), which classes were tested and which were not, and any known gaps (e.g., "oracle-manipulation exploit generation is the weak point; the harness detects but cannot construct the flash-loan PoC reliably"). Honesty about limitations is what makes the result credible — an over-claimed benchmark is dismissed, a fairly-scoped one is taken seriously. The audience you want respects the scope statement more than the score.

Where to publish, and why the benchmark IS the marketing asset

This one page goes on GitHub as the harness repo's README (benchmark table is the first thing a visitor sees), on LinkedIn as a post positioning the work as a falsifiable security claim, and on Deepthreat.ai as a demonstration asset.

The reason the benchmark result is the marketing asset — rather than a separate writeup — is that a scored, reproducible result against a named benchmark is the strongest claim you can make in security tooling. "I built a smart contract audit harness" is an assertion. "My harness scores 75% Detect / 67% Patch / 67% Exploit on a 4-vulnerability EVMbench subset, forked at block 19,200,000, methodology and evidence public" is a falsifiable claim. Anyone can re-run the fork at that block and check. The reproducibility is the marketing — it converts the work from a portfolio piece into evidence. This is the S05.3 and S12.1 argument applied to your own output.

This is a public portfolio artifact

The C2 capstone is the culmination of Course 2A. The published benchmark is what you show: a harness that runs three modes against a real target, produces a client-ready report with a tamper-evident evidence chain, and scores against a recognized benchmark. The score is falsifiable — anyone can run the same target and compare. The report is deliverable — a client could receive it as-is. And the one-page summary is publishable — it fits on a LinkedIn post and a GitHub README.

The harness is the engine. The benchmark is the proof. The published summary is the portfolio. This is what "Building AI Harnesses for Cybersecurity" produces: not a prompt, not a script, but a shippable, measurable, publishable security tool.

# Capstone C2 — Build a Smart Contract or Cloud Security Harness

**Course**: 2A — Building AI Harnesses for Cybersecurity
**Module**: C2 — Build a Smart Contract or Cloud Security Harness
**Duration**: 90 minutes
**Level**: Senior Engineer and above
**Prerequisites**: C1 complete. You have the harness architecture — scope middleware, memory, tools, evidence, triage, report — and you have benchmarked it. This capstone applies that architecture to a specialized domain and produces a publishable portfolio artifact.

> *This is a capstone. The teaching is design guidance for applying the C1 skeleton to smart contract or cloud security. The lab (artifact 07) is the build itself.*

---

## Learning Objectives

1. Choose a domain (EVMbench-aligned smart contract audit or cloud posture + red team) and produce an authorization and scope document.
2. Define the three modes the harness supports, the tool suite, the evidence schema, and the report format.
3. Implement the full pipeline for the chosen domain and run it against a real lab target.
4. Capture structured findings with full evidence records and generate a client-ready report.
5. Score the harness against the domain benchmark and produce a one-page results summary for LinkedIn, GitHub, and Deepthreat.ai.

---

# C2.1 — Design and Scoping

Fifteen minutes. You already have the architecture from C1. The design work here is domain-specific: choosing the track, defining the three modes, selecting the tools, and writing the authorization document.

## Choose your domain

Two tracks:

- **Smart contract audit harness (EVMbench-aligned).** Target is a set of smart contracts on a forked mainnet or a Damn Vulnerable DeFi (DVD) challenge. The harness runs three modes — Detect, Patch, Exploit — and is scored against an EVMbench subset. The deliverable is a client-ready audit report matching the structure top firms publish (scope, methodology, findings, severity, PoC, remediation).
- **Cloud posture + red team harness.** Target is an isolated AWS (or GCP/Azure) environment with deliberately misconfigured resources. The harness runs three modes — Detect, Remediate, Exploit — and is scored against a defined cloud posture benchmark (e.g., a CIS AWS Foundations subset or a custom control set). The deliverable is a posture report with findings, evidence, and a remediation roadmap.

Choose the track where you can stand up a real lab target in the build phase. For smart contracts, forked mainnet via Foundry or a DVD challenge is fast. For cloud, an isolated AWS account with Terraform-provisioned misconfigurations takes more setup but is the real thing.

### Selecting your EVMbench subset (smart contract track)

EVMbench ships 117 curated vulnerabilities across 40 real EVM repositories. For a capstone you do not run all 117 — you select a representative subset of 3 to 5 and score against those. The selection discipline is what makes the subset meaningful rather than cherry-picked: span the vulnerability classes, not the easy wins. Pick one from each of these classes so the subset exercises different detection and exploitation paths:

1. **Reentrancy** — the canonical class. Slither detects the checks-effects-interactions violation; the exploit must prove the reentrant call drains funds.
2. **Access control / privilege escalation** — a `public` function that should be `onlyRole`. Tests whether the harness reasons about authorization intent, not just visibility. The PoC is the unauthorized call itself.
3. **Price oracle manipulation** (DeFi-specific) — the class where purpose-built agents score 92% and general LLMs score 34% (S11.1, S12.2). Requires forked mainnet because the exploit depends on real protocol state (reserves, TWAP windows). This is the discriminative class.
4. **Arithmetic / invariant violation** — an integer issue or logic bug that breaks a protocol invariant. Tests the invariant-extraction harness from S11.2.

Three to five spanning these classes runs in the build window but exercises both static-detectable patterns (reentrancy, access control) and semantic-reasoning patterns (oracle, invariant). Document which vulnerabilities you selected and why — the selection rationale is part of the benchmark's defensibility. A reviewer who sees "we picked these four because they span the four major classes" trusts the subset; one who sees four reentrancy bugs does not.

## Define the three modes

The three modes are the spine of the harness. Each mode has its own tool chain and evidence shape:

**Smart contract (Detect / Patch / Exploit):**
- **Detect** — finds vulnerabilities using static analysis (Slither, Mythril), heuristic scaffolds, and LLM reasoning over reorganized source. Evidence: the finding with location and class.
- **Patch** — generates a fix that removes the vulnerability and preserves behavior. Verified by re-running detection (finding gone) and the test suite (behavior preserved). Evidence: the diff and the test results.
- **Exploit** — builds a working PoC that runs on forked mainnet and successfully exploits the target. Evidence: the PoC transaction and the state change proving exploitation.

**Cloud (Detect / Remediate / Exploit):**
- **Detect** — identifies posture violations and exposures using configuration analysis (AWS Config, Prowler), attack-surface discovery, and LLM reasoning. Evidence: the misconfigured resource and the violated control.
- **Remediate** — generates a remediation (an IAM policy fix, a security group change, a resource removal) that resolves the finding without breaking the application. Verified by re-running detection. Evidence: the remediation action and the post-fix control state.
- **Exploit** — attempts a controlled red-team action proving the exposure is real (e.g., assuming a role via an over-permissive policy, accessing an exposed S3 bucket). Evidence: the exploit chain and the access achieved.

A harness that only runs Detect is half a harness. The three modes together are what makes the benchmark meaningful — recall alone hides weaknesses in remediation and exploitation. This is the same argument from S11.1 and S12.1: detection without exploitation or patching is a hypothesis, not a delivered result.

## Define tool suite, evidence schema, and report format

The tool suite is domain-specific, but the pattern from C1 holds: each tool has a Pydantic input model, a scope check, a high-impact flag, and a run method. For smart contracts the suite is Slither, Mythril, a Foundry test runner, a forked-mainnet PoC builder, and an evidence recorder; for cloud it is a configuration scanner (Prowler/AWS Config), an IAM analyzer, an attack-surface mapper, a remediation applier, and an evidence recorder.

Each suite maps to its three modes. Smart-contract Detect uses Slither (reentrancy, integer issues, access control), Mythril (symbolic execution for deeper paths), and LLM reasoning over reorganized source (the context engineering from S11); Patch uses the Foundry runner; Exploit uses the forked-mainnet builder. Cloud Detect uses Prowler (CIS scanning) and the IAM analyzer (over-permissive policies, escalation paths); Remediate uses the applier (IAM policy fixes, security-group changes, resource removal); Exploit uses the attack-surface mapper to find a reachable exposure and attempts the red-team action. The evidence recorder captures each stage.

The evidence schema is the same tamper-evident chain from C1: finding links to evidence (with sha256 hash) links to tool call links to log line (trace ID). The report format is domain-specific — the smart contract report follows the S12 six-section structure (scope, methodology, findings table, severity, detailed findings, remediation roadmap); the cloud report follows a posture structure (scope, control framework, findings by category, evidence, remediation roadmap).

### Cascade verification (smart contract Patch mode)

The Patch mode is not "generate a diff and check it compiles." It is a cascade of verification gates from S11.4, each of which must pass before the patch is accepted:

1. **Detection clean.** Re-run Slither and Mythril against the patched source. The original finding must be gone — and no *new* findings may appear. A patch that removes the reentrancy but introduces an integer overflow fails.
2. **Test suite passes.** Run the Foundry test suite against the patched contract. Every existing test must pass — the patch preserved behavior. This is why you need a real test suite on the target; DVD and EVMbench repositories include them.
3. **Exploit fails.** Re-run the Exploit-mode PoC against the patched contract. This catches the trap from S11.4: a patch that fixes the *specific PoC transaction* but leaves the underlying flaw (e.g., blacklisting one attacker address while leaving the access-control gap open). If a second PoC can still exploit the patched code, the patch is incomplete.

All three pass, or the patch is rejected and the LLM retries within a budget. The cascade is what makes Patch quality a meaningful score rather than a formality.

## Write the authorization and scope document

For smart contracts, the authorization document defines: the contracts in scope (addresses and commit hashes), the forked-block number, what the harness is authorized to do (read, mutate in the fork, deploy PoCs), and the engagement boundary. For cloud, it defines: the AWS account ID, the regions in scope, the resources the harness may read and mutate, the actions it may take in Exploit mode (and the blast-radius limits), and the teardown procedure.

The authorization document is not a formality. For cloud especially, an exploit action against the wrong account or without blast-radius limits is a real incident. Write it before you build.

---

# C2.2 — Build and Run

Sixty minutes. Implement the full pipeline and run it against a real lab target.

## Implement the full pipeline for your chosen domain

The pipeline is the C1 harness with domain-specific tools and modes. The build order is the same: scope middleware, memory, tools, evidence, triage, report. The difference is the tools and the mode loop.

For smart contracts, the pipeline is:

1. Load the target contract source and the EVMbench metadata (expected vulnerabilities).
2. **Detect mode** — run Slither, Mythril, and the LLM reasoning over reorganized source. Collect candidate findings.
3. **Patch mode** — for each detected finding, generate a patch. Verify by re-running detection (finding gone) and the Foundry test suite (behavior preserved).
4. **Exploit mode** — for each detected finding, build a PoC on forked mainnet. Run it. Record success/failure.
5. Triage — filter false positives, confirm findings with evidence.
6. Report — generate the audit report.

### Foundry forked-mainnet setup (smart contract track)

Exploit mode and the third cascade gate both require a forked mainnet. Foundry does this natively:

```bash
forge test --fork-url $ARCHIVE_RPC_URL --fork-block-number $BLOCK -vvv
```

Two things must be pinned and recorded in the evidence: the `--fork-url` (an archive-capable RPC — Alchemy/Infura paid tier or a self-hosted archive node) and the `--fork-block-number`. Pinning the block is what makes the PoC reproducible — the forked state at block N is deterministic, so a reviewer re-running with the same RPC and block gets the same contract storage, oracle prices, and liquidity. A PoC that worked on "whatever block is current" is not reproducible and therefore not trustworthy as evidence. This is also why oracle-manipulation exploits belong in the subset: they require real protocol state (reserves, TWAP windows) that only exists on a fork — a toy contract cannot reproduce them.

For cloud, the pipeline is:

1. Load the target environment (account, regions) and the control framework (CIS subset).
2. **Detect mode** — run the configuration scanner and IAM analyzer. Collect posture violations.
3. **Remediate mode** — for each finding, generate and apply a remediation. Verify by re-running detection.
4. **Exploit mode** — for each exposure, attempt a controlled red-team action proving it is real.
5. Triage and report.

### The dual pipeline: posture and red team together (cloud track)

The cloud track's distinguishing decision is that Detect and Exploit are not independent — they are a dual pipeline where posture findings feed red team hypotheses. This integrates S09 (posture) and S10 (red team) into one harness, and it is what separates a cloud harness from a CSPM tool.

The posture scanner (Prowler, AWS Config) runs Detect and produces control violations — an over-permissive IAM role, a public S3 bucket, a security group open to 0.0.0.0/0. Each posture finding is a *hypothesis* that the exposure is real and reachable. The red team harness (S10's Pacu-based engine) validates each hypothesis: can an untrusted principal actually assume that role? Can an anonymous caller read that bucket?

When the red team succeeds, the finding is confirmed exploitable and its severity is elevated — a "public S3 bucket" is Medium from a configuration standpoint but Critical when the red team proves it holds sensitive data. When the red team fails (the bucket is public but empty, or the role exists but no untrusted principal can reach the assumption path), severity is reduced or the finding is marked informational. The dual pipeline turns static configuration data into validated risk — the S09-to-S10 bridge, and the cloud track's core value proposition.

### IAM graph construction from real AWS data

The IAM analyzer in Detect builds a privilege-escalation graph from live AWS data — the technique from S10.2. Enumerate principals (users, roles) and policies (managed and inline) via API calls (`list-roles`, `list-attached-role-policies`, `get-policy-version`, `list-role-policies`), then build a directed graph whose edges are the escalation primitives — `sts:AssumeRole`, `iam:PassRole`, `iam:CreatePolicy`, `iam:AttachRolePolicy`. A path from a starting principal to a privileged target is an escalation chain.

Build from live API calls, not Terraform state — live state is the truth, and drift between IaC and reality is itself a finding. The graph feeds Detect (the path is a posture violation) and Exploit (the red team traverses it). The evidence record captures the full chain: principal A assumes role B, B has `iam:PassRole` and passes role C to a service, C has admin — traceable link by link.

## Run against a real lab target

This is non-negotiable. A harness tested only on toy examples is a hypothesis. The lab target makes the benchmark falsifiable — anyone can run the same target and compare their scores. This is the repeatable-lab discipline from S05.3: isolated targets, seeded vulnerabilities, consistent scoring methodology, fixed compute envelope.

**Smart contract lab target options:**
- **Damn Vulnerable DeFi (DVD)** — deliberately vulnerable DeFi challenges with known solutions. Ideal because the protocols span real vulnerability classes (flash loan manipulation, price oracle abuse, governance attacks, lending-protocol liquidation logic). A harness that solves a DVD challenge has demonstrably found, understood, and exploited a real DeFi vulnerability.
- **Forked mainnet** — Foundry forks mainnet at a specific block and targets a known-vulnerable contract (e.g., a historical exploit before the fix). Highest fidelity — the state and contracts are real, and the PoC runs against the exact conditions at the time of the exploit. Requires an archive node or RPC provider supporting historical state.
- **EVMbench subset** — select 3-5 vulnerabilities using the subset-selection guidance above. Using EVMbench (the standard benchmark from S12) makes your scores directly comparable to published results.

**Cloud lab target options:**
- **Isolated AWS account** — provision misconfigured resources with Terraform (public S3 buckets, over-permissive IAM roles, exposed security groups, disabled logging). Real AWS APIs, real IAM evaluation, real resource behavior. Cost is low (a few dollars per run with prompt teardown) but setup requires an AWS account and Terraform. Use a dedicated account, never shared or production.
- **LocalStack** — a local AWS emulator for fast iteration without a real account. Some services are imperfectly emulated, so IAM-evaluation findings may not reproduce on real AWS. Use LocalStack for iteration, then validate on an isolated account before publishing the benchmark.

Run the full pipeline. Record every finding with its evidence. The success criterion is not "it ran" — it is "it produced confirmed findings with evidence for the known vulnerabilities/misconfigurations." A run that produces zero confirmed findings against a target with known bugs is a failed run. Debug the pipeline: did Detect find the bugs? Did the triage filter reject them erroneously? Did the evidence recorder fail to capture proof? Each failure point is a diagnostic.

## Capture structured findings with full evidence records

Every confirmed finding carries the full evidence chain from C1: finding ID, mode, tool call, raw output (PoC transaction, remediation diff, scanner output), sha256 hash, trace ID. A finding without evidence is an opinion; a finding with a tamper-evident chain is a result.

For smart contracts, an Exploit-mode evidence record contains the PoC transaction hash, the forked-block number, the state diff (before and after), and the Solidity code of the exploit contract — a reviewer can replay the PoC on the same forked block and verify the same state change. For cloud, it contains the API call chain (e.g., AssumeRole → ListObjects → GetObject), the credentials used (scoped to the isolated account), and the data accessed — a reviewer can verify the blast radius was contained.

The triage filter from C1 applies unchanged: candidate findings move to confirmed only with linked evidence and a passing confidence threshold; known false positives are filtered by rule (code, not LLM judgment); duplicates are merged. The state machine is domain-independent — what changes is the false-positive rule set (e.g., for smart contracts, a Slither finding on a test file is a false positive; for cloud, a Prowler finding on a resource in an excluded region is a false positive).

## Generate a client-ready report

Run the report generator on the confirmed findings. The report must be deliverable as-is:

- **Smart contract**: scope (contracts, commit, forked block), methodology (tools, modes), findings table (ID, title, severity, location, status), severity breakdown, detailed findings (description, impact, PoC, recommendation, evidence hash), remediation roadmap. Matches the S12 structure.
- **Cloud**: scope (account, regions, control framework), methodology (scanner, modes), findings by category (IAM, network, storage, logging), detailed findings (control violated, evidence, remediation applied), remediation roadmap.

The report is dual-output: JSON for machine consumption and HTML for the human reviewer.

---

# C2.3 — Benchmark and Publish

Fifteen minutes. This is the portfolio moment. The benchmark score and the client report distill into a single publishable page.

## Score against the domain benchmark

**Smart contract (EVMbench subset):** Score the three modes independently:
- **Detect recall** — of the N known vulnerabilities, how many did the harness find?
- **Patch quality** — for each detected finding, did the patch remove the vulnerability AND preserve behavior (test suite passes)?
- **Exploit success rate** — for each detected finding, did the harness build a working PoC on forked mainnet?

Report all three together. A harness scoring 85% Detect / 30% Patch / 30% Exploit is a detection engine, not an audit harness. The three scores together are the benchmark. The Patch quality gate has two parts: re-run detection on the patched code (the finding must be gone, and no new findings introduced), and run the Foundry test suite (all tests must pass — behavior preserved). A patch that removes the bug but breaks a test fails. A patch that fixes the specific PoC but leaves the underlying flaw fails (the third cascade gate catches this). Both gates must pass.

**Cloud (posture benchmark):** Score the three modes:
- **Detect recall** — of the N seeded misconfigurations, how many did the harness find?
- **Remediation success** — for each detected finding, did the remediation resolve it (re-scan clean) without breaking the application?
- **Exploit success** — for each exposure, did the harness achieve the red-team objective (e.g., assumed role, accessed data)?

The Remediation success gate also has two parts: re-scan the resource after applying the fix (the control must now pass), and verify the application still functions (the remediation did not break it — e.g., tightening an IAM policy did not revoke a permission the app needs). Over-remediation — removing all permissions to make the scan clean — breaks the application and fails the gate. The correct remediation tightens to least-privilege, not to zero-privilege.

### A worked example: scoring a smart contract run

Suppose your EVMbench subset is four vulnerabilities: a reentrancy bug, an access-control gap, an oracle-manipulation path, and an integer overflow. The harness produces:

| # | Vulnerability | Detect | Patch | Exploit |
|---|---------------|--------|-------|---------|
| 1 | Reentrancy | Found | Passed cascade (detection clean, tests pass, exploit now fails) | PoC drained funds on fork |
| 2 | Access control | Found | Passed cascade | PoC executed unauthorized call |
| 3 | Oracle manipulation | Found | Patch broke a test (retry budget exhausted) | PoC failed — flash-loan path not constructed |
| 4 | Integer overflow | Missed | — | — |

Scoring: Detect recall = 3 of 4 = 75%. Patch quality = 2 of 3 detected = 67%. Exploit success = 2 of 3 detected = 67%.

What the numbers tell you: detection missed the integer overflow (a Slither-detectable class — check the Slither wrapper or the confidence threshold). Patch failed on the oracle case because the patch broke a test. Exploit failed on the same case because flash-loan construction is the hardest PoC (it requires understanding reserves, routing, and timing — exactly where general LLMs score 34% per EVMbench). Each number points at a specific subsystem to improve. A single "detection rate" would hide all three failure modes. The cloud track's three scores follow the same diagnostic logic — a posture miss points at the scanner, a failed remediation points at the least-privilege logic, a failed exploit (e.g., an open port with no live service) points at the red-team's reachability check.

## Produce a one-page results summary

Distill the benchmark into a single page. This page has four required components:

**1. The benchmark table.** The three-mode scores, with the target and subset explicit:

```
[Harness Name] — Smart Contract Security Harness Benchmark

Target: EVMbench subset (4 vulnerabilities: reentrancy, access control,
        oracle manipulation, integer overflow) on forked mainnet
Forked block: 19,200,000
Date: 2026-07-09

| Mode        | Score | Detail                          |
|-------------|-------|---------------------------------|
| Detect      | 75%   | 3 of 4 vulnerabilities found    |
| Patch       | 67%   | 2 of 3 detected patched cleanly |
| Exploit     | 67%   | 2 of 3 detected exploited on fork |

Client report: [link to HTML report]
Evidence chain: [link to repo with evidence records + fork config]
```

**2. A methodology note.** Two to three sentences stating what was tested and how: the tools, the cascade gates applied, the forked-block pinning for reproducibility, and the compute envelope (model, approximate token cost — the S05.3 fixed-compute discipline). The methodology is what lets a reviewer judge whether the number is meaningful; without it the score is an assertion.

**3. An architecture diagram reference.** A link to or thumbnail of the three-mode pipeline (Detect → Triage → Patch/Exploit → Evidence → Report). It shows a reviewer that this is a system, not a script. Reference the S11 architecture if your diagram follows it.

**4. Honest limitations.** State what the benchmark does *not* claim: the subset size (4 vulnerabilities, not 117), which classes were tested and which were not, and any known gaps (e.g., "oracle-manipulation exploit generation is the weak point; the harness detects but cannot construct the flash-loan PoC reliably"). Honesty about limitations is what makes the result credible — an over-claimed benchmark is dismissed, a fairly-scoped one is taken seriously. The audience you want respects the scope statement more than the score.

### Where to publish, and why the benchmark IS the marketing asset

This one page goes on **GitHub** as the harness repo's README (benchmark table is the first thing a visitor sees), on **LinkedIn** as a post positioning the work as a falsifiable security claim, and on **Deepthreat.ai** as a demonstration asset.

The reason the benchmark result *is* the marketing asset — rather than a separate writeup — is that a scored, reproducible result against a named benchmark is the strongest claim you can make in security tooling. "I built a smart contract audit harness" is an assertion. "My harness scores 75% Detect / 67% Patch / 67% Exploit on a 4-vulnerability EVMbench subset, forked at block 19,200,000, methodology and evidence public" is a falsifiable claim. Anyone can re-run the fork at that block and check. The reproducibility is the marketing — it converts the work from a portfolio piece into evidence. This is the S05.3 and S12.1 argument applied to your own output.

## This is a public portfolio artifact

The C2 capstone is the culmination of Course 2A. The published benchmark is what you show: a harness that runs three modes against a real target, produces a client-ready report with a tamper-evident evidence chain, and scores against a recognized benchmark. The score is falsifiable — anyone can run the same target and compare. The report is deliverable — a client could receive it as-is. And the one-page summary is publishable — it fits on a LinkedIn post and a GitHub README.

The harness is the engine. The benchmark is the proof. The published summary is the portfolio. This is what "Building AI Harnesses for Cybersecurity" produces: not a prompt, not a script, but a shippable, measurable, publishable security tool.