AI Image Effects

AI Image Effects

Professional AI-powered image effects using Replicate models - Illusion Diffusion, FLUX Fill Pro, and Meta SAM-2.

Overview

Three powerful AI effects for image transformation:

Quick Start

1. Configure Replicate API

# Set API token
export REPLICATE_API_TOKEN="r8_xxxxxxxxxxxxx"

# Check balance
python scripts/check_credit.py

2. Run Effects

# Illusion Diffusion
from src.illusion_diffusion import IllusionDiffusion

illusion = IllusionDiffusion()
result = illusion.apply(
    image_path="input.png",
    prompt="spiral pattern",
    strength=1.5
)

# FLUX Fill Pro
from src.flux_fill_pro import FluxFillPro

flux = FluxFillPro()
result = flux.replace_accessory(
    image_path="milady.png",
    accessory_type="hat",
    new_description="red baseball cap"
)

# SAM Detector
from src.sam_detector import SAMDetector

sam = SAMDetector()
bbox = sam.detect_accessory(
    image_path="milady.png",
    accessory_type="glasses"
)

Effect 1: Illusion Diffusion

What It Does

Creates optical illusions and pattern effects using ControlNet. The AI reshapes the image while maintaining recognizability from a distance.

Use Cases

• Spiral patterns - Make image appear as spiral
• QR codes - Embed QR codes in artwork
• Hidden text - Text visible from afar
• Geometric patterns - Hexagons, triangles, waves
• Artistic effects - Creative distortions

Basic Usage

from src.illusion_diffusion import IllusionDiffusion

illusion = IllusionDiffusion()

# Simple spiral effect
result = illusion.apply(
    image_path="milady.png",
    prompt="spiral optical illusion"
)
# Cost: $0.006

Advanced Parameters

result = illusion.apply(
    image_path="milady.png",
    prompt="concentric circles optical illusion",
    control_guidance=1.5,        # 0.5-5.0 (how strong the effect)
    controlnet_conditioning=1.2, # 0.5-2.0 (pattern strength)
    steps=50,                    # 20-100 (quality vs speed)
    guidance_scale=7.5           # 0-20 (prompt adherence)
)

Parameter Guide:

• control_guidance: Higher = stronger illusion effect
• controlnet_conditioning: Higher = more visible pattern
• steps: More steps = higher quality (but slower)
• guidance_scale: Higher = follows prompt more closely

Pattern Ideas

# Spiral
illusion.apply(image, "fibonacci spiral pattern")

# Geometric
illusion.apply(image, "hexagonal tessellation")

# Wave
illusion.apply(image, "concentric wave pattern")

# QR Code
illusion.apply(image, "QR code pattern", control_guidance=2.0)

# Text
illusion.apply(image, "hidden text 'WAGMI' optical illusion")

Cost Optimization

# Lower quality (faster, cheaper)
result = illusion.apply(
    image,
    prompt="spiral",
    steps=20,              # Faster
    control_guidance=1.0   # Standard strength
)
# Still costs $0.006 per image

# Batch processing
results = illusion.batch_apply(
    images=["img1.png", "img2.png", "img3.png"],
    prompt="spiral pattern"
)
# Cost: $0.006 × 3 = $0.018

Effect 2: FLUX Fill Pro

What It Does

AI inpainting and accessory replacement. Intelligently fills/replaces parts of an image with new content matching the description.

Use Cases

• Replace accessories - Swap hats, glasses, earrings
• Change clothes - Modify shirts, dresses
• Add elements - Insert new objects
• Remove elements - Inpaint to remove objects
• Style transfer - Change artistic style of regions

Basic Usage

from src.flux_fill_pro import FluxFillPro

flux = FluxFillPro()

# Replace hat
result = flux.replace_accessory(
    image_path="milady.png",
    accessory_type="hat",
    new_description="red baseball cap"
)
# Cost: $0.05

Accessory Types

Supports 8 predefined accessory categories:

ACCESSORY_TYPES = {
    "hat": (300, 180, 80),         # (y, height, width %)
    "glasses": (420, 60, 50),
    "earrings": (480, 80, 70),
    "necklace": (640, 120, 60),
    "scarf": (550, 150, 70),
    "clothes": (700, 400, 90),
    "background": (0, 1250, 100),
    "other": None  # Manual bbox
}

Advanced Usage

# Custom bounding box
result = flux.replace_region(
    image_path="milady.png",
    bbox=(100, 200, 300, 400),  # (x, y, width, height)
    prompt="cool futuristic visor"
)

# With mask image
result = flux.inpaint(
    image_path="milady.png",
    mask_path="mask.png",        # White = replace, Black = keep
    prompt="blue sunglasses"
)

# Batch replace multiple accessories
result = flux.batch_replace(
    image_path="milady.png",
    replacements=[
        {"type": "hat", "description": "cowboy hat"},
        {"type": "glasses", "description": "round sunglasses"}
    ]
)
# Cost: $0.05 × 2 = $0.10

Quality Parameters

result = flux.replace_accessory(
    image_path="milady.png",
    accessory_type="hat",
    new_description="red beanie",
    strength=0.8,           # 0-1 (how much to change)
    guidance=7.5,           # Prompt adherence
    steps=30,               # Generation steps
    seed=42                 # Reproducibility
)

Best Practices

1. Be specific in descriptions

   # ❌ Vague
   "hat"
   
   # ✅ Specific
   "red baseball cap with white logo"
   

2. Use consistent lighting

   "blue sunglasses with same lighting as original image"
   

3. Match art style

   "anime-style pink beret matching the character's aesthetic"
   

Effect 3: SAM Detector

What It Does

Meta's Segment Anything Model (SAM-2) for precise object detection and bounding box extraction. Used to automatically find accessories before replacement.

Use Cases

• Auto-detect accessories - Find hats, glasses automatically
• Precise segmentation - Get exact object boundaries
• Multi-object detection - Detect all objects in image
• Smart cropping - Crop to detected objects
• Integration with FLUX - Auto-detect then replace

Basic Usage

from src.sam_detector import SAMDetector

sam = SAMDetector()

# Detect accessory
bbox = sam.detect_accessory(
    image_path="milady.png",
    accessory_type="hat"
)
# Returns: (x, y, width, height) or None
# Cost: <$0.01

Supported Accessories

ACCESSORY_CATEGORIES = {
    "hat": ["帽子", "hat", "cap", "beanie", "beret"],
    "glasses": ["眼镜", "glasses", "sunglasses", "spectacles"],
    "earrings": ["耳环", "earring", "earrings"],
    "necklace": ["项链", "necklace", "chain"],
    "scarf": ["围巾", "scarf", "bandana"],
    "clothes": ["衣服", "shirt", "dress", "top"],
    "face_decoration": ["面部装饰", "sticker", "mark"],
    "overlay": ["特效", "effect", "overlay"]
}

Smart Detection

# Auto-detect with smart positioning
result = sam.smart_detect(
    image_path="milady.png",
    accessory_type="glasses",
    use_position_hints=True  # Uses expected position for better accuracy
)

# Detect with confidence threshold
bbox, confidence = sam.detect_with_confidence(
    image_path="milady.png",
    accessory_type="hat",
    min_confidence=0.7
)

Caching System

SAM results are cached for 7 days to save costs:

# First call - runs SAM ($0.01)
bbox1 = sam.detect_accessory("milady.png", "hat")

# Second call - uses cache (FREE)
bbox2 = sam.detect_accessory("milady.png", "hat")

# Clear cache if needed
sam.clear_cache()

# Check cache stats
stats = sam.get_cache_stats()
# Returns: {"hits": 150, "misses": 50, "savings": "$0.50"}

Integration with FLUX

# Combo: SAM detection + FLUX replacement
from src.sam_detector import SAMDetector
from src.flux_fill_pro import FluxFillPro

sam = SAMDetector()
flux = FluxFillPro()

# Step 1: Detect hat automatically
bbox = sam.detect_accessory("milady.png", "hat")

# Step 2: Replace with FLUX
if bbox:
    result = flux.replace_region(
        image_path="milady.png",
        bbox=bbox,
        prompt="red baseball cap"
    )
# Total cost: <$0.01 + $0.05 = ~$0.05

Advanced Features

# Get all detected objects
all_objects = sam.detect_all_objects("milady.png")
# Returns: [{"bbox": (x,y,w,h), "label": "hat", "confidence": 0.95}, ...]

# Get segmentation mask (not just bbox)
mask = sam.get_segmentation_mask("milady.png", "glasses")
# Returns: PIL Image (binary mask)

# Batch detection
bboxes = sam.batch_detect(
    images=["img1.png", "img2.png"],
    accessory_type="hat"
)

Complete Workflow Example

Example 1: Smart Accessory Replacement

from src.sam_detector import SAMDetector
from src.flux_fill_pro import FluxFillPro

sam = SAMDetector()
flux = FluxFillPro()

# Auto-detect and replace
def smart_replace(image_path, accessory_type, new_description):
    """Detect accessory with SAM, replace with FLUX"""

    # Step 1: Detect
    print(f"Detecting {accessory_type}...")
    bbox = sam.detect_accessory(image_path, accessory_type)

    if not bbox:
        print(f"No {accessory_type} detected, using default position")
        # Fallback to default
        return flux.replace_accessory(
            image_path, accessory_type, new_description
        )

    # Step 2: Replace
    print(f"Replacing {accessory_type} with: {new_description}")
    result = flux.replace_region(
        image_path=image_path,
        bbox=bbox,
        prompt=new_description
    )

    return result

# Use it
result = smart_replace(
    "milady_5050.png",
    "glasses",
    "cool futuristic visor with neon blue glow"
)
result.save("output.png")

Example 2: Multi-Effect Pipeline

from src.illusion_diffusion import IllusionDiffusion
from src.flux_fill_pro import FluxFillPro

illusion = IllusionDiffusion()
flux = FluxFillPro()

# Step 1: Replace accessories
print("Replacing hat...")
img = flux.replace_accessory(
    "milady.png",
    "hat",
    "red baseball cap"
)
img.save("step1.png")

# Step 2: Apply illusion effect
print("Applying spiral effect...")
final = illusion.apply(
    "step1.png",
    prompt="spiral optical illusion"
)
final.save("final.png")

print("Total cost: $0.05 + $0.006 = $0.056")

Example 3: Batch Processing

from src.flux_fill_pro import FluxFillPro

flux = FluxFillPro()

# Process multiple images
images = ["milady_1.png", "milady_2.png", "milady_3.png"]
accessories = ["hat", "glasses", "hat"]
descriptions = [
    "cowboy hat",
    "heart-shaped sunglasses",
    "blue beanie"
]

results = []
for img, acc, desc in zip(images, accessories, descriptions):
    print(f"Processing {img}...")
    result = flux.replace_accessory(img, acc, desc)
    results.append(result)

# Cost: $0.05 × 3 = $0.15

Configuration

Replicate Config

Located in: config/replicate_config.py

# Model versions
MODELS = {
    "illusion": "monster-labs/control_v1p_sd15_qrcode_monster:...",
    "flux_fill": "black-forest-labs/flux-fill-pro",
    "sam": "meta/sam-2:..."
}

# Default parameters
ILLUSION_DEFAULTS = {
    "control_guidance": 1.5,
    "steps": 50
}

FLUX_DEFAULTS = {
    "strength": 0.8,
    "guidance": 7.5,
    "steps": 30
}

Cost Management

# Check current balance
python scripts/check_credit.py

# Output:
# Replicate Balance: $10.50
# Recent usage: $2.30 (last 24h)

Rate Limiting

# Built-in rate limiting to avoid API overload
flux = FluxFillPro(max_concurrent=3)  # Max 3 simultaneous requests

Cost Calculator

Estimate Costs

from src.cost_calculator import CostCalculator

calc = CostCalculator()

# Single operation
cost = calc.estimate(
    effect="flux_fill",
    count=1
)
# Returns: $0.05

# Batch operation
cost = calc.estimate_batch([
    {"effect": "illusion", "count": 10},
    {"effect": "flux_fill", "count": 5},
    {"effect": "sam", "count": 20}
])
# Returns: $0.06 + $0.25 + $0.20 = $0.51

Monthly Budget Planning

# Plan monthly usage
monthly_plan = {
    "illusion": 100,      # 100 images
    "flux_fill": 50,      # 50 replacements
    "sam": 200            # 200 detections
}

cost = calc.monthly_estimate(monthly_plan)
# Returns: $0.60 + $2.50 + $2.00 = $5.10/month

Error Handling

Common Issues

1. Insufficient credits

try:
    result = flux.replace_accessory(...)
except InsufficientCreditsError:
    print("Please add credits to Replicate account")

2. Model timeout

# Increase timeout for slow models
flux = FluxFillPro(timeout=300)  # 5 minutes

3. Invalid bounding box

# Validate bbox before using
bbox = sam.detect_accessory(image, "hat")
if bbox and flux.validate_bbox(bbox):
    result = flux.replace_region(image, bbox, prompt)

Best Practices

1. Use SAM before FLUX - Auto-detection is more accurate than hardcoded positions
2. Cache SAM results - Saves 50-70% on detection costs
3. Batch when possible - Process multiple images in one session
4. Monitor spending - Check balance regularly
5. Test with low steps - Use steps=20 for testing, steps=50 for production
6. Be specific with prompts - Better descriptions = better results

Advanced Tips

Combine Multiple Effects

# Illusion + FLUX pipeline
def create_custom_illusion(image_path):
    # Replace accessories first
    img = flux.replace_accessory(image_path, "hat", "cool hat")

    # Then apply illusion
    final = illusion.apply(img, "spiral pattern")

    return final

Quality vs Cost Trade-off

# Production quality (higher cost, slower)
result = flux.replace_accessory(
    image, "hat", "red cap",
    steps=50,
    guidance=10.0
)

# Draft quality (lower cost, faster)
result = flux.replace_accessory(
    image, "hat", "red cap",
    steps=20,
    guidance=7.0
)
# Both cost $0.05, but draft is 2x faster

Related Documentation

• ILLUSION_GUIDE.md - Detailed illusion patterns
• FLUX_FILL_GUIDE.md - FLUX best practices
• SAM_DETECTOR_GUIDE.md - SAM detection guide
• COST_CALCULATOR.md - Budget planning

Related Skills

• milady-meme-generator - Generate base images
• lark-bot-integration - Use in Lark bot

Total Costs Summary:

• Illusion Diffusion: $0.006 per image
• FLUX Fill Pro: $0.05 per image
• SAM Detection: <$0.01 per detection
• Combined (SAM + FLUX): ~$0.05 per image