YARA Performance Guidelines

YARA Performance Guidelines

Complete reference for writing efficient YARA rules. Based on YARA Performance Guidelines v1.6 (Feb 2025).

"Think of YARA as a two-step process: first, searching for all patterns listed in the strings, and second, evaluating the conditions. You can't use well-formed conditions to make up for poorly chosen strings." — Wesley Shields

How YARA Scans (The 4 Steps)

1. Compiling the Rules

YARA extracts atoms (4-byte substrings) from defined strings to feed the Aho-Corasick automaton. YARA picks atoms cleverly to avoid too many matches.

Example: For strings <?php, GET, POST, /assert[\t ]{0,100}\(/, YARA might pick:

• <?ph
• GET
• POST
• sser (from "assert")

2. Aho-Corasick Search

Scans files for those atoms using a prefix tree. Matches are handed to the bytecode engine.

3. Bytecode Engine

Verifies full string matches. If sser matches, YARA checks for a prefix and t continuation, then validates the regex [\t ]{0,100}\(. This avoids running slow regex over entire files.

4. Condition Evaluation

Checks additional rule logic. Expensive operations (like math.entropy) only run if prior conditions pass (short-circuit evaluation).

Atoms (The Most Important Factor)

YARA extracts up to 4-byte substrings called "atoms" from strings. These atoms drive the fast Aho-Corasick search.

Good Atoms

YARA picks the best available atoms:

/abc.*cde/           → atoms: "abc" or "cde" (prefers "abc")
/(one|two)three/     → atoms: "thre" preferred (longer, more unique)
{ 00 00 00 00 [1-4] 01 02 03 04 }  → uses "01 02 03 04" (avoids common zeros)
{ 01 02 [1-4] 01 02 03 04 }        → prefers "01 02 03 04" over "01 02"

Bad Atoms (Avoid These)

// Too short (< 4 bytes of unique content)
$a = "MZ"
$b = { 4D 5A }

// Repeating/uniform content
$c = "AAAAAAAAAAAAAAAA"
$d = "\x00\x20\x00\x20\x00\x20"  // wide spaces
$e = { 00 00 00 00 [1-2] FF FF [1-2] 00 00 00 00 }

// No fixed anchor (worst - forces naïve matching)
$f = /\w.*\d/
$g = /[0-9]+\n/

Better Alternatives

// For short header checks, use uint functions
uint16(0) == 0x5A4D  // instead of "MZ" at 0

// Add context to extend atoms
$better = "MZ\x90\x00\x03"  // 4+ bytes

// Regex MUST have 4+ byte anchor
$anchored = /mshta\.exe http:\/\/[a-z0-9\.\/]{3,70}\.hta/

String Selection Best Practices

Too Short Strings

Any string < 4 bytes will probably appear in many files OR as uniform content in XORed files.

Uniform Content

These cause "too many matches" errors:

$s1 = "22222222222222222222222222222222222222222222222222222222222222"
$s2 = "\x00\x20\x00\x20\x00\x20\x00\x20"  // wide formatted spaces

Error message:

error scanning yara-killer.dat: string "$mz" in rule "shitty_mz" caused too many matches

String Modifiers & Atom Count

LOW atom count (1 atom):

$s1 = "cmd.exe"               // ascii only (default)
$s2 = "cmd.exe" ascii         // same as $s1
$s3 = "cmd.exe" wide          // UTF-16 only
$s4 = "cmd.exe" ascii wide    // 2 atoms
$s5 = { 63 6d 64 2e 65 78 65 }  // hex equivalent

HIGH atom count (up to 16 atoms for 4-byte strings):

$s5 = "cmd.exe" nocase        // all case combinations: "Cmd.", "cMd.", "cmD." ...

⚠️ Use nocase carefully. If you need specific variations, use regex: $re = /[Pp]assword/

Alternation Issues

Alternations generate short atoms:

// Generates short atoms - SLOW
$re = /(a|b)cde/
$hex = { C7 C3 00 (31 | 33) }

// Better: Split into separate strings
$re1 = /acde/
$re2 = /bcde/
$hex1 = { C7 C3 00 31 }
$hex2 = { C7 C3 00 33 }

Regular Expressions

Use regex only when necessary. Regex evaluation is slower and consumes significant memory.

Regex Quantifiers

Quantifier Behavior

Anchored suffix (one possible beginning): YARA matches the longest possible. .* and .+ can lead to huge matches.

$re1 = /Tom.{0,2}/      // finds "Tomxx" in "Tomxx"
$re2 = /.{0,2}Tom/      // finds "Tom", "xTom", "xxTom" in "xxTom"

Multiple shorter matches can cross the limit and cause "too many matches" errors.

Email Regex Example

AVOID (match multiple times):

/[-a-z0-9._%+]*@[-a-z0-9.]{2,10}\.[a-z]{2,4}/
/[-a-z0-9._%+]+@[-a-z0-9.]{2,10}\.[a-z]{2,4}/

USE (match subset):

/[-a-z0-9._%+]@[-a-z0-9.]{2,10}\.[a-z]{2,4}/
OR
/@[-a-z0-9.]{2,10}\.[a-z]{2,4}/

Use Offsets Instead of Regex

// SLOW: Greedy regex
$ = /exec.*\/bin\/sh/

// FAST: String + offset check
strings:
  $exec = "exec"
  $sh   = "/bin/sh"
condition:
  $exec and $sh and @exec < @sh

Regex Anchors

Longer anchors = better performance:

// BAD: No anchor, greedy
$s1 = /http:\/\/[.]*\.hta/

// BETTER: Upper bound
$s1 = /http:\/\/[a-z0-9\.\/]{3,70}\.hta/

// BEST: Long fixed prefix
$s1 = /mshta\.exe http:\/\/[a-z0-9\.\/]{3,70}\.hta/

Conditions & Short-Circuit Evaluation

YARA evaluates left-to-right and stops at first FALSE. Order matters significantly.

Reordering Impact

// No improvement (similar cost)
$string1 and $string2 and uint16(0) == 0x5A4D

// SLOW: Expensive first
math.entropy(0, filesize) > 7.0 and uint16(0) == 0x5A4D

// FAST: Cheap check first
uint16(0) == 0x5A4D and math.entropy(0, filesize) > 7.0

Loop Optimization

// SLOW: Loops over entire file
for all i in (1..filesize) : ($a at i)

// BETTER: Short-circuit with cheap check
$mz at 0 and for all i in (1..filesize) : ( whatever )

// BEST: Limit iterations
$mz at 0 and filesize < 100KB and for all i in (1..filesize) : ( whatever )

Loop Iterations Warning

// PROBLEM: $a is too common, #a can be huge
strings:
    $a = {00 00}
condition:
    for all i in (1..#a) : (@a[i] < 10000)

YARA 3.10+ optimized loops:

for all i in (0..100): (false)   // stops after first iteration
for any i in (0..100): (true)    // stops after first iteration

⚠️ Regex Does NOT Short-Circuit

Regex strings are always evaluated regardless of condition order:

// This is SLOW for ALL files, not just small ones
strings:
  $expensive_regex = /\$[a-z0-9_]+\(/ nocase
conditions:
  filesize < 200 and $expensive_regex

Module Alternatives

Modules parse entire files before evaluation, increasing scan time.

Magic Module (Avoid)

Not available on Windows. Slows down scanning.

// SLOW: Uses magic module
import "magic"
rule gif_2 {
  condition:
    magic.mime_type() == "image/gif"
}

// FAST: Header check
rule gif_1 {
  condition:
    (uint32be(0) == 0x47494638 and uint16be(4) == 0x3961) or
    (uint32be(0) == 0x47494638 and uint16be(4) == 0x3761)
}

PE Module Alternative

// SLOW: Parses full PE
import "pe"
condition: pe.is_pe

// FAST: Header check only
condition: uint16(0) == 0x5A4D

Fixing "Too Many Matches" & Slow Scanning

Diagnostic Steps

1. Check regex quantifiers: .*, .+, .*?
2. Add upper bounds: {x,y} not {x,}
3. Check large ranges: {1,300000} → reduce
4. Reduce hex wildcards: minimize jump ranges
5. Split alternations: /(a|b)/ → two strings
6. Add word boundaries: fullword, \b

Note on Condition Changes

While condition ordering helps performance, it will NOT fix "too many matches" errors. Those are caused by string patterns, not condition logic.

Metadata Memory Usage

All metadata is loaded into RAM. You can test this by inserting 100,000 hashes and checking YARA's RAM usage.

If memory-constrained, remove unneeded metadata before scanning.

Quick Reference Table

Video Tutorial

@herrcore has created a video covering these topics: Introduction Into YARA - Writing Efficient YARA Rules