Memory Analyzer

Memory Analyzer Skill

Overview

Comprehensive memory analysis skill for detecting memory leaks, analyzing heap dumps, and optimizing memory usage across Python, Java, JavaScript, and C/C++ applications.

Capabilities

1. Memory Leak Detection

• Identify growing memory patterns
• Track object retention
• Find reference cycles
• Detect zombie objects

2. Heap Analysis

• Heap dump analysis (Java, Python, Node.js)
• Object allocation tracking
• Memory profiling
• Retained memory calculation

3. Cross-Language Support

• Python: memory_profiler, objgraph, tracemalloc
• Java: Eclipse MAT, YourKit, VisualVM
• JavaScript/Node.js: Chrome DevTools, heap snapshots
• C/C++: Valgrind, AddressSanitizer

Python Memory Analysis

Using memory_profiler

from memory_profiler import profile

@profile
def memory_intensive_function():
    # Line-by-line memory tracking
    large_list = [i for i in range(10000000)]
    return large_list

# Run with: python -m memory_profiler script.py

Using tracemalloc (Built-in)

import tracemalloc

tracemalloc.start()

# Your code here
current, peak = tracemalloc.get_traced_memory()
print(f"Current: {current / 1024 / 1024:.2f}MB, Peak: {peak / 1024 / 1024:.2f}MB")

tracemalloc.stop()

Using objgraph (Reference Cycles)

import objgraph

# Find memory leaks
objgraph.show_most_common_types(limit=10)

# Track object growth
objgraph.show_growth(limit=5)

# Visualize reference chain
objgraph.show_backrefs([obj], filename='refs.png')

Java Memory Analysis

Eclipse MAT (Memory Analyzer Tool)

# Generate heap dump
jmap -dump:live,format=b,file=heap.hprof <pid>

# Analyze with MAT
# 1. Open heap.hprof in Eclipse MAT
# 2. Run Leak Suspects Report
# 3. Check Dominator Tree
# 4. Find paths to GC roots

VisualVM Heap Analysis

# Capture heap dump in VisualVM
# Analyze:
# - Classes by instance count
# - Objects by retained size
# - OQL queries for custom analysis

JavaScript/Node.js Memory Analysis

Chrome DevTools

// Take heap snapshot
// 1. Open Chrome DevTools > Memory tab
// 2. Take snapshot
// 3. Compare snapshots to find leaks
// 4. Look for detached DOM nodes

// Example: Finding memory leaks
let leakyArray = [];
setInterval(() => {
    leakyArray.push(new Array(1000000));
    console.log(`Memory leak: ${leakyArray.length} arrays`);
}, 1000);

Node.js Heap Profiling

# Generate heap snapshot
node --expose-gc --inspect script.js

# In Chrome DevTools:
# chrome://inspect -> Take heap snapshot

C/C++ Memory Analysis

Valgrind (Memcheck)

# Detect memory leaks
valgrind --leak-check=full --show-leak-kinds=all ./program

# Output analysis:
# - Definitely lost: Real memory leaks
# - Indirectly lost: Lost due to other leaks
# - Possibly lost: Pointer manipulation
# - Still reachable: Not freed but accessible

AddressSanitizer

# Compile with ASan
gcc -fsanitize=address -g program.c -o program

# Run and detect memory errors
./program

Integration Scripts

memory_leak_detector.py

Automated memory leak detection:

#!/usr/bin/env python3
import tracemalloc
import time
import psutil
import os

def monitor_memory(duration=60, interval=5):
    """Monitor memory usage over time"""
    process = psutil.Process(os.getpid())
    samples = []

    tracemalloc.start()

    for i in range(duration // interval):
        mem_info = process.memory_info()
        current, peak = tracemalloc.get_traced_memory()

        samples.append({
            'time': i * interval,
            'rss_mb': mem_info.rss / 1024 / 1024,
            'vms_mb': mem_info.vms / 1024 / 1024,
            'python_current_mb': current / 1024 / 1024,
            'python_peak_mb': peak / 1024 / 1024
        })

        time.sleep(interval)

    # Detect leak (>20% growth)
    if samples:
        growth = (samples[-1]['rss_mb'] - samples[0]['rss_mb']) / samples[0]['rss_mb']
        if growth > 0.20:
            print(f"⚠️  MEMORY LEAK DETECTED: {growth*100:.1f}% growth")
        else:
            print(f"✓ Memory stable: {growth*100:.1f}% growth")

    return samples

heap_compare.sh

Java heap dump comparison:

#!/bin/bash
# Compare two heap dumps to find memory leaks

DUMP1=$1
DUMP2=$2

echo "Analyzing heap dumps..."
echo "Before: $DUMP1"
echo "After: $DUMP2"

# Use jhat to compare
jhat -J-mx4g $DUMP2 &
JHAT_PID=$!

echo "Open http://localhost:7000 to compare heap dumps"
echo "Look for classes with growing instance counts"

Common Memory Issues

1. Memory Leaks

Symptoms:

• Steadily increasing memory usage
• OutOfMemoryError (Java)
• Process killed by OOM killer (Linux)

Causes:

• Event listeners not removed
• Global variables accumulating data
• Caches without eviction policy
• Unclosed database connections
• Circular references (Python GC usually handles this)

2. Excessive Allocations

Symptoms:

• High GC pressure (Java)
• Frequent memory allocations
• Poor performance

Causes:

• Creating objects in tight loops
• String concatenation in loops
• Large intermediate collections
• Inefficient data structures

3. Retained Objects

Symptoms:

• Memory not freed after use
• High retained heap size

Causes:

• Static references
• Singleton caches
• Long-lived collections holding references
• Session objects not cleaned up

Best Practices

1. Baseline First: Measure normal memory usage
2. Profile Under Load: Realistic workload scenarios
3. Take Multiple Snapshots: Compare before/after
4. Focus on Retained Size: Not just shallow size
5. Check Reference Chains: Why objects aren't freed
6. Monitor in Production: Use low-overhead tools
7. Set Memory Limits: Detect leaks early
8. Use Weak References: For caches and listeners

Memory Optimization Strategies

Python

# Use generators instead of lists
def read_large_file(filename):
    with open(filename) as f:
        for line in f:  # Generator
            yield line.strip()

# Use __slots__ to reduce object size
class OptimizedClass:
    __slots__ = ['field1', 'field2']

# Use weak references for caches
import weakref
cache = weakref.WeakValueDictionary()

Java

// Use StringBuilder for concatenation
StringBuilder sb = new StringBuilder();
for (String s : strings) {
    sb.append(s);
}

// Use object pooling for frequently created objects
// Implement proper equals/hashCode for Set/Map keys
// Close resources with try-with-resources
try (Connection conn = getConnection()) {
    // Use connection
}

Requirements

# Python
pip install memory-profiler objgraph psutil

# Java
# Eclipse MAT: https://www.eclipse.org/mat/
# VisualVM: Included with JDK

# C/C++
sudo apt-get install valgrind

# Node.js
# Chrome DevTools (built-in with Chrome)

Metrics to Track

• RSS (Resident Set Size): Physical memory used
• VMS (Virtual Memory Size): Total virtual memory
• Heap Size: Java heap, Python heap
• Object Count: Instances per class
• Retained Size: Memory kept alive by object
• Allocation Rate: MB/second allocated
• GC Frequency: Collections per minute