12 min read
Jataka Engineering
January 15, 2024

Why Static Analysis Can't Catch
Runtime Errors

Your CI pipeline runs PMD, SonarQube, or Clayton. They catch naming convention violations, security vulnerabilities, and code smells. You feel safe. Then a developer merges code with a SOQL query inside a for loop. Static analysis flags it as a "potential issue." You ship it anyway. Production crashes at 2:00 AM with 127 SOQL queries against a limit of 100.

The Fundamental Problem

Static analysis tools scan your source code as text. They parse the abstract syntax tree, apply pattern-matching rules, and flag violations. This is excellent for catching:

  • Unused variables and dead code paths
  • Security vulnerabilities like SOQL injection
  • Naming convention violations
  • Cyclomatic complexity thresholds

But static analysis cannot predict runtime behavior because it doesn't execute your code. It doesn't know:

  • How many records are in your production org
  • What other triggers fire when you update a record
  • How long your nested loops will actually take
  • Whether your DML operations conflict with Setup objects

A Concrete Example

Consider this code that passes static analysis but crashes in production:

AccountTriggerHandler.clsPasses Static Analysis
// This code passes static analysis
// It crashes in production

public void processAccounts(List<Id> accountIds) {
    for (Id accId : accountIds) {
        // Static analysis: "SOQL in loop - potential issue"
        // Runtime profiling: "127 queries executed, limit is 100"
        List<Contact> contacts = [
            SELECT Id, Name, Email
            FROM Contact
            WHERE AccountId = :accId
        ];
        
        for (Contact c : contacts) {
            c.Email = c.Email.toLowerCase();
        }
        update contacts;
    }
}

Static analysis says: "SOQL inside for loop - potential issue."

Runtime profiling says: "127 SOQL queries executed against a limit of 100. Transaction will fail in production."

Static vs Runtime: The Scorecard

Error Type
Static Analysis
Runtime Profiling
Winner
SOQL 101
Flags SOQL in loop (maybe)
Measures 127 queries vs 100 limit
Runtime wins
DML 151
Flags DML in loop (maybe)
Measures 187 DML vs 150 limit
Runtime wins
CPU Timeout
No detection possible
Measures 12,847ms vs 10,000ms
Runtime wins
Data Skew
No detection possible
Analyzes 52,847 child records
Runtime wins
Mixed DML
No detection possible
Detects Setup/non-Setup conflict
Runtime wins

The Solution: Runtime Profiling

Runtime profiling executes your code in an isolated environment with production-scale data. It measures what actually happens:

  • 1
    Actual SOQL count — Not "potential SOQL in loop" but "127 queries executed"
  • 2
    Actual CPU time — Not "nested loops detected" but "12,847ms consumed"
  • 3
    Actual DML operations — Not "DML in loop" but "187 statements vs 150 limit"
  • 4
    Actual data model impact — Not "possible contention" but "52,847 child records on parent"

What This Means for Your Team

Static analysis isn't dead—it's just incomplete. You still need it for code quality, security scanning, and style enforcement. But you also need runtime profiling for the things static analysis can't see.

Best practice for Salesforce DevSecOps:

Run static analysis (PMD, Clayton, SonarQube) for code quality.
Run runtime profiling (Jataka) for Governor Limit safety.
Use both. Your production environment will thank you.

See runtime profiling in action

Book a Demo

Watch Jataka catch the errors your static analysis tools miss.