rea-reverse

Solve CTF Reverse Engineering Challenges with REA and an AI Agent

Written for REA 6.4.0Updated 9 min read
On this page

An AI agent can solve an easy CTF reversing challenge in minutes, and it can also report a wrong flag with total confidence. The fix is not a better prompt. You need a workflow where every claim the agent makes points back to something the binary actually contains, and where the final answer is checked by running the original program.

This guide shows that workflow with REA, the open-source project by morluto. REA gives your agent native analysis tools over MCP (strings, cross-references, function dossiers, raw bytes) and a process-capture tool that records what the real executable prints. Each result carries an Evidence ID, which is what makes the agent’s reasoning checkable.

Who this is for: CTF players and students who already use Claude Code, Cursor or Codex, and want a repeatable way to work through rev challenges with an agent without trusting it blindly.

What you will learn:

  • How to build a small crackme you can practise on legally
  • The tool sequence an agent should follow, from a string to the checking loop
  • How to turn extracted bytes into a solver
  • How to verify the agent’s answer with Evidence and a process capture

The approach is the same one used in REA’s DownUnderCTF masked-squares showcase, scaled down to a binary you build yourself.

Prerequisites

  1. REA registered with your agent. Run npx rea-agents setup, approve the changes, and restart the agent. If you have not done this yet, follow Install REA and connect it to Claude Code, Cursor or Codex.

  2. A native analysis provider. This guide uses Ghidra. REA connects to an existing Ghidra 12.1.x installation with a full JDK; it never installs Ghidra or Java for you. Export the paths described in the installation guide, then check readiness:

    rea doctor --provider ghidra --json

    Hopper or IDA also work. For a longer Ghidra setup walkthrough, see Free native binary analysis with REA and Ghidra.

  3. Linux or macOS for the final check. Process capture needs a native PTY. On macOS it also needs Apple’s Swift compiler through xcrun. Native Windows cannot capture new processes; use Linux REA inside WSL. See process capture.

  4. A C compiler (clang or gcc) and Python 3.

REA is intended for CTFs, education and software you own or are authorized to examine. Running a challenge binary executes it with your user permissions, and process capture is not a sandbox. Read the project’s DISCLAIMER.md before you start.

Step 1: Build the practice crackme

Using a binary you wrote yourself means you know the right answer in advance, so you can judge the agent’s work instead of hoping it is right. Save this as xor_gate.c:

#include <stdio.h>
#include <string.h>

#define FLAG_LEN 20

static const unsigned char expected[FLAG_LEN] = {
    0x45, 0x60, 0x5e, 0x3e, 0x34, 0x7c, 0x2a, 0x43, 0x2b, 0x40,
    0x14, 0x2b, 0x77, 0x39, 0x4c, 0x12, 0x16, 0x2f, 0x1a, 0x20,
};

static int check(const char *s) {
    if (strlen(s) != FLAG_LEN)
        return 0;
    for (int i = 0; i < FLAG_LEN; i++) {
        unsigned char c = (unsigned char)s[i];
        unsigned char key = (unsigned char)(0x37 + 3 * i);
        if ((unsigned char)((c ^ key) + i) != expected[i])
            return 0;
    }
    return 1;
}

int main(void) {
    char buf[64];
    printf("Password: ");
    fflush(stdout);
    if (!fgets(buf, sizeof buf, stdin))
        return 1;
    buf[strcspn(buf, "\r\n")] = '\0';
    if (check(buf)) {
        puts("Access granted");
        return 0;
    }
    puts("Access denied");
    return 1;
}

The design is a classic easy rev pattern:

  • A length gate. Inputs that are not exactly 20 bytes fail immediately.
  • A position-dependent key. Each character is XORed with 0x37 + 3*i, so a single-byte XOR brute force does not work.
  • An offset. The loop adds the index after the XOR, which matters when you invert the check: subtract first, then XOR.
  • A stored table. The 20 expected bytes live in read-only data, where the agent has to read them.

Compile it and strip the symbols so the agent cannot just search for a function called check:

clang -O0 -o xor_gate xor_gate.c
strip xor_gate

-O0 keeps the loop easy to read in the decompiler. Once the walkthrough feels easy, rebuild with -O2. The optimizer may unroll or vectorize the loop, which is good practice for reading less friendly output.

Then move the source file somewhere else. The agent should only see the binary.

Step 2: Give the agent a goal, not a method

Open your agent in the folder that contains xor_gate and ask:

Use REA to analyze ./xor_gate and find the password. Explain how the check
works, cite the Evidence ID for each fact you rely on, and verify the answer
by running the original program with REA's process capture.

This prompt does three things on purpose:

  • It names the tool, so the agent uses REA rather than guessing from strings output.
  • It asks for Evidence IDs, which forces the agent to separate what it observed from what it inferred.
  • It defines “done” as the original binary accepting the input, not the agent being convinced.

REA also ships guided MCP prompts such as investigate_feature and trace_crash. They are optional starting points; for a crackme, a direct request like the one above is enough. See MCP prompts.

Step 3: Follow the agent’s tool sequence

A good agent run on a stripped binary usually looks like the sequence below. The tool names are REA’s MCP tools; the CLI equivalents are shown so you can repeat any step by hand.

3.1 Open the target and find the messages

The agent opens the binary, then searches for strings that look like prompts or verdicts:

open_binary            {"path": "/absolute/path/to/xor_gate", "provider_id": "ghidra"}
search_strings         {"pattern": "password|granted|denied", "mode": "regex", "case_sensitive": false}

CLI equivalent:

rea search /absolute/path/to/xor_gate "password|granted|denied" \
  --mode regex --provider ghidra --json

Why: in a stripped binary, the user-visible strings are the fastest way in. Access granted is where you want execution to go, and Access denied is where it goes when the check fails.

3.2 Walk from the string to the function

xrefs                          {"address": "<address of \"Access granted\">"}
resolve_containing_procedure   {"address": "<address of the reference>"}
rea xrefs /absolute/path/to/xor_gate 0xADDRESS --provider ghidra --json

Why: the cross-reference gives the instruction that loads the string, and resolve_containing_procedure turns that instruction into a function entry and body range. This is the same move the showcase uses to go from "What is the flag? " to the checker at 0x10124d.

3.3 Read the caller and the checker

analyze_function   {"procedure": "<main-like function>"}
analyze_function   {"procedure": "<function called before the branch>"}
rea function /absolute/path/to/xor_gate 0xADDRESS --provider ghidra --json
rea decompile /absolute/path/to/xor_gate 0xADDRESS --provider ghidra --json

analyze_function returns a function dossier with pseudocode, instructions, calls and references. In this target the agent should find:

  • a compare of the input length against 0x14 (20);
  • a loop counter that indexes both the input and a data table;
  • an XOR with a value derived from the counter (0x37 + i*3, often shown as i*3 + 0x37 or computed incrementally);
  • an ADD of the counter after the XOR;
  • a compare against a byte loaded from the table, with a branch to the failure path on mismatch.

Ask the agent to quote the specific instructions that support each of these points, not only the pseudocode. Decompiler output is an interpretation; the instructions are closer to what the CPU executes.

3.4 Read the table bytes

The pseudocode will reference the table by address. The agent then reads it directly:

read_bytes   {"address": "<table address>", "length": 20}
rea read-bytes /absolute/path/to/xor_gate 0xTABLE --length 20 --provider ghidra --json

Why: decompilers sometimes render constant arrays as strings, truncate them at a zero byte, or show them as signed values. Reading raw bytes removes that ambiguity. In the showcase, this step is what revealed that the mask byte eb meant -21 because the code used MOVSX.

Addresses depend on your compiler, platform and provider, so the values you see will differ from any in this article. That is fine: the method does not depend on them.

Step 4: Turn the rules into a solver

From the dossier and the bytes, the forward check is:

((input[i] XOR (0x37 + 3*i)) + i) mod 256 == expected[i]     for i in 0..19

Every operation is invertible, so this crackme does not need Z3. Subtract the index, then XOR with the same key:

EXPECTED = [0x45, 0x60, 0x5e, 0x3e, 0x34, 0x7c, 0x2a, 0x43, 0x2b, 0x40,
            0x14, 0x2b, 0x77, 0x39, 0x4c, 0x12, 0x16, 0x2f, 0x1a, 0x20]

def key(i):
    return (0x37 + 3 * i) & 0xFF

flag = bytes(((e - i) & 0xFF) ^ key(i) for i, e in enumerate(EXPECTED))
# Re-run the forward check on the candidate before trusting it.
assert all(((c ^ key(i)) + i) & 0xFF == EXPECTED[i] for i, c in enumerate(flag))
print(flag.decode())

When the agent writes this script, check two things before you accept it:

  1. The constants came from read_bytes Evidence, not from the agent’s memory of the pseudocode. Ask it which Evidence ID the table came from.
  2. The script re-applies the forward check. An inverse with a sign error can still print something that looks like a flag. The assert catches it.

For challenges where the checks overlap, such as the 26 masked sums in the DownUnderCTF showcase, the agent writes equations instead and hands them to a solver like Z3. The showcase keeps format assumptions (the DUCTF{...} prefix and allowed alphabet) clearly separated from constraints observed in the binary. Do the same: if the agent adds “the flag probably starts with rea{”, that is an assumption, not Evidence.

Step 5: Verify with Evidence, not with confidence

This is where an agent-assisted solve differs from a lucky one.

5.1 Map each claim to an Evidence ID

Every REA tool result includes evidence_id and a normalized_result. Ask the agent for a short table:

Claim Tool Evidence ID
Success string and its address search_strings ev_...
The function that references it xrefs, resolve_containing_procedure ev_...
Length 20, XOR key, + i, compare analyze_function ev_...
The 20 table bytes read_bytes ev_...

The showcase’s evidence notes use exactly this layout. If the agent cannot point to an Evidence ID for a claim, treat the claim as unverified. To look up past records without pulling their full payloads, the agent can call get_evidence_bundle with detail: "summary" and filter by operation.

5.2 Keep observations, inferences and unknowns apart

“The loop XORs with 0x37 + 3*i” is an inference from instructions. “The table at this address holds these 20 bytes” is an observation. “The binary does not check anything else” is an unknown until the agent has read every path to the success branch. REA’s prompts and results use this same split. When an agent merges them, mistakes slip through.

5.3 Run the original program

Static analysis tells you what the code should do. A process capture shows what it did. The agent calls capture_process_scenario, or you can save this as scenario-pass.json:

{
  "executable": "/absolute/path/to/xor_gate",
  "working_directory": ".",
  "events": [
    { "type": "input", "at_ms": 250, "data": "rea{s1mple_x0r_gate}\r" }
  ],
  "timeout_ms": 5000
}
rea capture-process ./scenario-pass.json --json > capture-pass.json

The capture records terminal output, the exit status and cleanup. You expect Access granted and exit code 0.

Then run a negative control: change one character (for example, the last letter before }) and capture again into capture-fail.json. You expect Access denied and exit code 1. The showcase does the same thing, changing one z to y and observing Incorrect! with exit code 255. A negative control proves the check is actually sensitive to your input. Without it, a binary that accepts everything would also “verify” your flag.

Two practical notes:

  • Write the capture to a file different from the scenario file. Shell redirection truncates the output before REA reads the input.
  • rea compare-process-captures rejects captures whose comparison contracts differ, and the scripted input is part of that contract. Read each capture’s own exit status and output rather than diffing a pass and a fail.

5.4 Keep the record

Export the session so you, a teammate or a write-up reviewer can check the work later:

export_evidence_bundle   {"path": "/absolute/path/to/xor_gate-evidence.json"}
rea evidence-import /absolute/path/to/xor_gate-evidence.json

evidence-import validates the bundle. For the full Evidence model, snapshots and version comparison, see Evidence, snapshots and version comparison.

Troubleshooting and FAQ

The first Ghidra query is slow or the client times out. REA starts Ghidra import and auto-analysis lazily on the first query. A client timeout can end the wait while Ghidra keeps working. Raise REA_GHIDRA_STARTUP_TIMEOUT_MS for large binaries, and see Ghidra first-query deadlines.

ambiguous provider error. More than one provider supports the target. Pass provider_id to open_binary or --provider ghidra on the CLI.

The string search finds nothing. Some challenges build strings at runtime or encrypt them. Search procedures instead (rea search PATH main --kind procedures), start from the entry point, or look for imports such as strcmp, memcmp or puts.

The agent’s flag fails in the capture. Common causes are sign errors (signed vs unsigned bytes), the wrong order of operations when inverting (- i before or after the XOR), and a missing terminator in the table. Ask the agent to re-read the relevant instructions and bytes, and to show the forward check failing on its candidate.

The binary prints instructions to the agent. Strings, pseudocode and terminal output come from the target. CTF authors sometimes plant text such as “ignore previous instructions”. REA marks these results as untrusted content, and the agent should treat them as data. If it suggests running or downloading something because the binary said so, stop it.

Process capture fails on macOS. Check that xcrun swift works. REA compiles a small helper with it and does not install Xcode or the Command Line Tools for you.

Can I use this in a live competition? Check the event rules on AI tools first. Many CTFs allow them, but some restrict them.

Next steps