Task: CTF Challenge. You are solving a capture-the-flag reverse engineering challenge. The goal is finding the flag.
Approach
Be targeted and efficient. CTF binaries are usually small, purpose-built, and contain a clear solve path. Don't over-analyze — find the check/validation function, understand the constraint, solve it.
Workflow
get_binary_info + list_functions — orient yourself, find main or entry (batch these)
decompile_function on main — identify the input path and validation logic
- Trace the check function: usually a comparison, hash check, or transformation chain
- Identify the algorithm: XOR, custom cipher, hash, math constraints, maze/game, VM-based
search_strings for flag format strings (CTF{, flag{, HTB{, etc.)
- Solve: extract the key/flag directly, reverse the transformation, or write a solver
Common Patterns
- Flag format strings visible in
list_strings or search_strings
- Input validation concentrated in a single function
- XOR with static key — extract key and data, XOR to get flag
- Base64 or custom encoding — identify the table, decode
- Constraint satisfaction — extract constraints, use z3 via
execute_python
- Anti-debug checks (ptrace, IsDebuggerPresent) guarding the real logic — bypass or ignore
- Multi-stage: unpacking → decryption → flag check
- VM-based: custom bytecode interpreter — map opcodes, trace execution, extract constraints
Solving Strategies
Direct extraction: If the flag is compared byte-by-byte or XOR'd with
a known key, extract both operands and compute the flag directly.
Constraint solving: For complex validation (many conditions, polynomial
checks, matrix transforms), extract constraints and write a z3 solver:
from z3 import *
s = Solver()
flag = [BitVec(f'c{i}', 8) for i in range(N)]
# Add constraints from decompiled validation...
s.add(...)
if s.check() == sat:
m = s.model()
print(''.join(chr(m[c].as_long()) for c in flag))
Transformation reversal: If the input goes through a series of
reversible transforms (XOR, rotate, shuffle, substitution), reverse
each step in order.
Tips
- If you find encrypted/encoded data, try to reverse the algorithm from the decompiled code
- For constraint solving, write and execute a Python script with
execute_python
- Focus on the solve path — don't enumerate every function or produce threat reports
- Check
xrefs_to on comparison/validation functions to find where the flag is checked
- Look at string xrefs — flag-related strings often lead directly to the validation logic
- If stuck, check for hardcoded keys or constants near the comparison code
1---2name: ctf-challenge3description: Capture-the-flag reverse engineering — find the flag efficiently4---5Task: CTF Challenge. You are solving a capture-the-flag reverse engineering challenge. The goal is finding the flag.67## Approach89Be targeted and efficient. CTF binaries are usually small, purpose-built, and contain a clear solve path. Don't over-analyze — find the check/validation function, understand the constraint, solve it.1011## Workflow12131. `get_binary_info` + `list_functions` — orient yourself, find main or entry (batch these)142. `decompile_function` on main — identify the input path and validation logic153. Trace the check function: usually a comparison, hash check, or transformation chain164. Identify the algorithm: XOR, custom cipher, hash, math constraints, maze/game, VM-based175. `search_strings` for flag format strings (CTF{, flag{, HTB{, etc.)186. Solve: extract the key/flag directly, reverse the transformation, or write a solver1920## Common Patterns2122- **Flag format strings** visible in `list_strings` or `search_strings`23- **Input validation** concentrated in a single function24- **XOR with static key** — extract key and data, XOR to get flag25- **Base64 or custom encoding** — identify the table, decode26- **Constraint satisfaction** — extract constraints, use z3 via `execute_python`27- **Anti-debug checks** (ptrace, IsDebuggerPresent) guarding the real logic — bypass or ignore28- **Multi-stage**: unpacking → decryption → flag check29- **VM-based**: custom bytecode interpreter — map opcodes, trace execution, extract constraints3031## Solving Strategies3233**Direct extraction:** If the flag is compared byte-by-byte or XOR'd with34a known key, extract both operands and compute the flag directly.3536**Constraint solving:** For complex validation (many conditions, polynomial37checks, matrix transforms), extract constraints and write a z3 solver:38```python39from z3 import *40s = Solver()41flag = [BitVec(f'c{i}', 8) for i in range(N)]42# Add constraints from decompiled validation...43s.add(...)44if s.check() == sat:45 m = s.model()46 print(''.join(chr(m[c].as_long()) for c in flag))47```4849**Transformation reversal:** If the input goes through a series of50reversible transforms (XOR, rotate, shuffle, substitution), reverse51each step in order.5253## Tips5455- If you find encrypted/encoded data, try to reverse the algorithm from the decompiled code56- For constraint solving, write and execute a Python script with `execute_python`57- Focus on the solve path — don't enumerate every function or produce threat reports58- Check `xrefs_to` on comparison/validation functions to find where the flag is checked59- Look at string xrefs — flag-related strings often lead directly to the validation logic60- If stuck, check for hardcoded keys or constants near the comparison code