Nextjs Modal Integration

name: nextjs-modal-integration description: Modal.com integration patterns for Next.js applications. PROACTIVELY activate for: (1) Next.js + Modal backend setup, (2) AI inference from Next.js (LLMs, image generation), (3) Video/audio processing backends, (4) Heavy compute offloading from Vercel, (5) GPU workloads for Next.js apps, (6) Webhook integration between Next.js and Modal, (7) File upload processing, (8) Background job processing, (9) Serverless AI API endpoints, (10) Next.js + Modal authentication patterns. Provides: Architecture patterns, API route integration, webhook handling, file upload workflows, CORS configuration, warm container patterns, streaming responses, and production-ready examples.

Quick Reference

When to Use This Skill

Use for Next.js + Modal integration:

• AI inference that's too heavy for Edge/Vercel Functions
• Video/audio processing with FFmpeg
• Background jobs exceeding Vercel's 60s timeout
• GPU workloads (image generation, LLMs, embeddings)
• Cost-effective scaling for burst compute

Architecture principle: Next.js handles UI/auth/routing, Modal handles heavy compute.

Next.js + Modal.com Integration (2025)

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                    Next.js (Vercel)                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │   Pages/    │  │    API      │  │    Server           │ │
│  │   App       │  │   Routes    │  │    Actions          │ │
│  │   Router    │  │             │  │                     │ │
│  └──────┬──────┘  └──────┬──────┘  └──────────┬──────────┘ │
└─────────┼────────────────┼───────────────────┼─────────────┘
          │                │                   │
          └────────────────┼───────────────────┘
                           │ HTTPS
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                    Modal.com                                │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │   FastAPI   │  │    GPU      │  │    Background       │ │
│  │   Endpoint  │  │   Functions │  │    Jobs             │ │
│  │             │  │   (A100)    │  │    (.spawn())       │ │
│  └─────────────┘  └─────────────┘  └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Modal Backend Setup

Basic FastAPI Endpoint

# modal_backend/app.py
import modal
from datetime import datetime

app = modal.App("nextjs-backend")

image = (
    modal.Image.debian_slim(python_version="3.11")
    .pip_install("fastapi", "pydantic")
)

@app.function(image=image)
@modal.concurrent(max_inputs=100, target_inputs=50)
@modal.asgi_app()
def api():
    """FastAPI endpoint for Next.js frontend"""
    from fastapi import FastAPI, HTTPException, Depends
    from fastapi.middleware.cors import CORSMiddleware
    from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
    from pydantic import BaseModel, Field

    web_app = FastAPI(title="Next.js Backend API")

    # CORS configuration for Next.js
    web_app.add_middleware(
        CORSMiddleware,
        allow_origins=[
            "http://localhost:3000",           # Next.js dev
            "https://*.vercel.app",            # Vercel preview
            "https://yourdomain.com",          # Production
        ],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )

    # Simple API key auth
    security = HTTPBearer()
    API_KEY = "your-secret-key"  # Use modal.Secret in production

    def verify_token(creds: HTTPAuthorizationCredentials = Depends(security)):
        if creds.credentials != API_KEY:
            raise HTTPException(status_code=401, detail="Invalid API key")
        return creds.credentials

    # === Endpoints ===

    class ProcessRequest(BaseModel):
        data: str = Field(..., min_length=1)
        options: dict = {}

    class ProcessResponse(BaseModel):
        result: str
        processed_at: str

    @web_app.post("/process", response_model=ProcessResponse)
    def process_endpoint(
        req: ProcessRequest,
        token: str = Depends(verify_token)
    ):
        # Your processing logic here
        result = f"Processed: {req.data}"

        return ProcessResponse(
            result=result,
            processed_at=datetime.utcnow().isoformat()
        )

    @web_app.get("/health")
    def health():
        return {"status": "healthy", "timestamp": datetime.utcnow().isoformat()}

    return web_app

Deploy with:

modal deploy modal_backend/app.py
# Returns: https://your-workspace--nextjs-backend-api.modal.run

Next.js API Route Integration

Basic API Route (App Router)

// app/api/process/route.ts
import { NextRequest, NextResponse } from 'next/server';

const MODAL_API_URL = process.env.MODAL_API_URL!;
const MODAL_API_KEY = process.env.MODAL_API_KEY!;

export async function POST(req: NextRequest) {
  try {
    const body = await req.json();

    const response = await fetch(`${MODAL_API_URL}/process`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${MODAL_API_KEY}`,
      },
      body: JSON.stringify({
        data: body.data,
        options: body.options || {},
      }),
    });

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`Modal API error: ${response.status} - ${error}`);
    }

    const result = await response.json();
    return NextResponse.json(result);

  } catch (error) {
    console.error('Processing error:', error);
    return NextResponse.json(
      { error: 'Processing failed' },
      { status: 500 }
    );
  }
}

Environment Variables

# .env.local
MODAL_API_URL=https://your-workspace--nextjs-backend-api.modal.run
MODAL_API_KEY=your-secret-key

AI Image Generation Example

Modal Backend

# modal_backend/image_gen.py
import modal

app = modal.App("image-generator")

image = (
    modal.Image.debian_slim(python_version="3.11")
    .pip_install(
        "fastapi",
        "torch",
        "diffusers",
        "transformers",
        "accelerate",
        "pydantic",
    )
)

models_volume = modal.Volume.from_name("sd-models", create_if_missing=True)

@app.cls(
    image=image,
    gpu="A100-40GB",
    volumes={"/models": models_volume},
    min_containers=1,  # Keep warm for fast response
    max_containers=5,
    container_idle_timeout=300,
)
class ImageGenerator:
    @modal.enter()
    def setup(self):
        import torch
        from diffusers import StableDiffusionXLPipeline

        print("Loading SDXL model...")
        self.pipe = StableDiffusionXLPipeline.from_pretrained(
            "stabilityai/stable-diffusion-xl-base-1.0",
            torch_dtype=torch.float16,
            cache_dir="/models",
        )
        self.pipe.to("cuda")
        print("Model ready!")

    @modal.method()
    def generate(
        self,
        prompt: str,
        negative_prompt: str = "",
        width: int = 1024,
        height: int = 1024,
        steps: int = 30,
    ) -> bytes:
        """Generate image and return as PNG bytes"""
        import io

        image = self.pipe(
            prompt=prompt,
            negative_prompt=negative_prompt,
            width=width,
            height=height,
            num_inference_steps=steps,
        ).images[0]

        buffer = io.BytesIO()
        image.save(buffer, format="PNG")
        return buffer.getvalue()


@app.function(image=image)
@modal.concurrent(max_inputs=50)
@modal.asgi_app()
def api():
    from fastapi import FastAPI, HTTPException
    from fastapi.middleware.cors import CORSMiddleware
    from fastapi.responses import Response
    from pydantic import BaseModel, Field

    web_app = FastAPI()

    web_app.add_middleware(
        CORSMiddleware,
        allow_origins=["http://localhost:3000", "https://*.vercel.app"],
        allow_methods=["*"],
        allow_headers=["*"],
    )

    class GenerateRequest(BaseModel):
        prompt: str = Field(..., min_length=1, max_length=1000)
        negative_prompt: str = ""
        width: int = Field(1024, ge=512, le=2048)
        height: int = Field(1024, ge=512, le=2048)
        steps: int = Field(30, ge=20, le=50)

    @web_app.post("/generate")
    def generate_endpoint(req: GenerateRequest):
        generator = ImageGenerator()

        try:
            image_bytes = generator.generate.remote(
                prompt=req.prompt,
                negative_prompt=req.negative_prompt,
                width=req.width,
                height=req.height,
                steps=req.steps,
            )
            return Response(content=image_bytes, media_type="image/png")
        except Exception as e:
            raise HTTPException(status_code=500, detail=str(e))

    return web_app

Next.js API Route

// app/api/generate-image/route.ts
import { NextRequest, NextResponse } from 'next/server';

const MODAL_API_URL = process.env.MODAL_IMAGE_GEN_URL!;

export async function POST(req: NextRequest) {
  try {
    const body = await req.json();

    const response = await fetch(`${MODAL_API_URL}/generate`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        prompt: body.prompt,
        negative_prompt: body.negativePrompt || '',
        width: body.width || 1024,
        height: body.height || 1024,
        steps: body.steps || 30,
      }),
    });

    if (!response.ok) {
      throw new Error(`Generation failed: ${response.statusText}`);
    }

    const imageBuffer = await response.arrayBuffer();

    return new NextResponse(imageBuffer, {
      headers: {
        'Content-Type': 'image/png',
        'Cache-Control': 'public, max-age=31536000, immutable',
      },
    });
  } catch (error) {
    console.error('Image generation error:', error);
    return NextResponse.json({ error: 'Generation failed' }, { status: 500 });
  }
}

React Component

// components/ImageGenerator.tsx
'use client';

import { useState } from 'react';

export default function ImageGenerator() {
  const [prompt, setPrompt] = useState('');
  const [imageUrl, setImageUrl] = useState('');
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState('');

  const generateImage = async () => {
    setLoading(true);
    setError('');

    try {
      const response = await fetch('/api/generate-image', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ prompt }),
      });

      if (!response.ok) {
        throw new Error('Generation failed');
      }

      const blob = await response.blob();
      const url = URL.createObjectURL(blob);
      setImageUrl(url);
    } catch (err) {
      setError(err instanceof Error ? err.message : 'Unknown error');
    } finally {
      setLoading(false);
    }
  };

  return (
    <div className="max-w-2xl mx-auto p-6">
      <h1 className="text-3xl font-bold mb-6">AI Image Generator</h1>

      <textarea
        className="w-full p-4 border rounded-lg mb-4 min-h-[100px]"
        placeholder="Describe the image you want to generate..."
        value={prompt}
        => setPrompt(e.target.value)}
      />

      <button
        className="w-full bg-blue-600 text-white py-3 px-6 rounded-lg font-medium
                   disabled:opacity-50 disabled:cursor-not-allowed
                   hover:bg-blue-700 transition-colors"
       
        disabled={loading || !prompt.trim()}
      >
        {loading ? 'Generating...' : 'Generate Image'}
      </button>

      {error && (
        <div className="mt-4 p-4 bg-red-100 text-red-700 rounded-lg">
          {error}
        </div>
      )}

      {imageUrl && (
        <div className="mt-6">
          <img
            src={imageUrl}
            alt="Generated image"
            className="w-full rounded-lg shadow-lg"
          />
        </div>
      )}
    </div>
  );
}

Long-Running Jobs with Webhooks

For jobs exceeding Vercel's 60-second timeout, use webhooks.

Modal Backend with Webhooks

# modal_backend/jobs.py
import modal
import httpx

app = modal.App("job-processor")

image = modal.Image.debian_slim().pip_install("fastapi", "httpx", "pydantic")

@app.function(image=image, timeout=3600)  # 1 hour max
def process_long_job(job_id: str, data: dict, callback_url: str):
    """Long-running job that calls back when complete"""
    import time

    # Simulate long processing
    print(f"Processing job {job_id}...")
    time.sleep(60)  # Your actual processing here

    result = {"job_id": job_id, "status": "completed", "output": "processed data"}

    # Callback to Next.js webhook
    httpx.post(
        callback_url,
        json=result,
        headers={"X-Webhook-Secret": "your-webhook-secret"},
        timeout=30,
    )

    return result


@app.function(image=image)
@modal.asgi_app()
def api():
    from fastapi import FastAPI, BackgroundTasks
    from pydantic import BaseModel
    import uuid

    web_app = FastAPI()

    class JobRequest(BaseModel):
        data: dict
        callback_url: str

    @web_app.post("/submit-job")
    def submit_job(req: JobRequest):
        job_id = str(uuid.uuid4())

        # Spawn async job (returns immediately)
        process_long_job.spawn(job_id, req.data, req.callback_url)

        return {
            "job_id": job_id,
            "status": "processing",
            "message": "Job submitted successfully"
        }

    return web_app

Next.js Webhook Handler

// app/api/webhooks/job-complete/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { prisma } from '@/lib/prisma'; // Your database client

const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET!;

export async function POST(req: NextRequest) {
  // Verify webhook signature
  const signature = req.headers.get('X-Webhook-Secret');
  if (signature !== WEBHOOK_SECRET) {
    return NextResponse.json({ error: 'Invalid signature' }, { status: 401 });
  }

  try {
    const body = await req.json();
    const { job_id, status, output } = body;

    // Update job in database
    await prisma.job.update({
      where: { id: job_id },
      data: {
        status,
        output,
        completedAt: new Date(),
      },
    });

    // Optional: Send notification to user
    // await sendNotification(job.userId, `Job ${job_id} completed!`);

    return NextResponse.json({ received: true });
  } catch (error) {
    console.error('Webhook error:', error);
    return NextResponse.json({ error: 'Processing failed' }, { status: 500 });
  }
}

Job Submission from Next.js

// app/api/jobs/submit/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { prisma } from '@/lib/prisma';
import { auth } from '@/lib/auth';

const MODAL_API_URL = process.env.MODAL_JOBS_URL!;
const APP_URL = process.env.NEXT_PUBLIC_APP_URL!;

export async function POST(req: NextRequest) {
  const session = await auth();
  if (!session?.user) {
    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
  }

  try {
    const body = await req.json();

    // Create job record
    const job = await prisma.job.create({
      data: {
        userId: session.user.id,
        status: 'pending',
        input: body,
      },
    });

    // Submit to Modal
    const response = await fetch(`${MODAL_API_URL}/submit-job`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        data: body,
        callback_url: `${APP_URL}/api/webhooks/job-complete`,
      }),
    });

    if (!response.ok) {
      throw new Error('Failed to submit job');
    }

    return NextResponse.json({
      jobId: job.id,
      status: 'processing',
    });
  } catch (error) {
    console.error('Job submission error:', error);
    return NextResponse.json({ error: 'Submission failed' }, { status: 500 });
  }
}

File Upload Processing

Modal Backend for File Processing

# modal_backend/files.py
import modal

app = modal.App("file-processor")

image = modal.Image.debian_slim().pip_install(
    "fastapi",
    "python-multipart",
    "pillow",
)

@app.function(image=image, timeout=300)
@modal.asgi_app()
def api():
    from fastapi import FastAPI, UploadFile, File, HTTPException
    from fastapi.responses import Response
    from PIL import Image
    import io

    web_app = FastAPI()

    @web_app.post("/resize-image")
    async def resize_image(
        file: UploadFile = File(...),
        width: int = 800,
        height: int = 600,
    ):
        if not file.content_type.startswith('image/'):
            raise HTTPException(400, "File must be an image")

        contents = await file.read()

        # Process image
        img = Image.open(io.BytesIO(contents))
        img = img.resize((width, height), Image.Resampling.LANCZOS)

        # Return processed image
        buffer = io.BytesIO()
        img.save(buffer, format="PNG")

        return Response(
            content=buffer.getvalue(),
            media_type="image/png",
        )

    return web_app

Next.js File Upload Handler

// app/api/resize/route.ts
import { NextRequest, NextResponse } from 'next/server';

const MODAL_API_URL = process.env.MODAL_FILES_URL!;

export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const file = formData.get('file') as File;
    const width = formData.get('width') || '800';
    const height = formData.get('height') || '600';

    if (!file) {
      return NextResponse.json({ error: 'No file provided' }, { status: 400 });
    }

    // Forward to Modal
    const modalFormData = new FormData();
    modalFormData.append('file', file);

    const response = await fetch(
      `${MODAL_API_URL}/resize-image?width=${width}&height=${height}`,
      {
        method: 'POST',
        body: modalFormData,
      }
    );

    if (!response.ok) {
      throw new Error('Processing failed');
    }

    const imageBuffer = await response.arrayBuffer();

    return new NextResponse(imageBuffer, {
      headers: { 'Content-Type': 'image/png' },
    });
  } catch (error) {
    console.error('Resize error:', error);
    return NextResponse.json({ error: 'Processing failed' }, { status: 500 });
  }
}

Streaming LLM Responses

Modal Backend with Streaming

# modal_backend/llm.py
import modal

app = modal.App("llm-api")

image = modal.Image.debian_slim().pip_install(
    "fastapi",
    "transformers",
    "torch",
    "accelerate",
    "sse-starlette",
)

@app.cls(image=image, gpu="A100", min_containers=1)
class LLMServer:
    @modal.enter()
    def setup(self):
        from transformers import AutoTokenizer, AutoModelForCausalLM
        import torch

        self.tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
        self.model = AutoModelForCausalLM.from_pretrained(
            "meta-llama/Llama-2-7b-chat-hf",
            torch_dtype=torch.float16,
        ).to("cuda")

    @modal.method()
    def generate_stream(self, prompt: str, max_tokens: int = 512):
        """Generator that yields tokens one at a time"""
        from transformers import TextIteratorStreamer
        from threading import Thread

        inputs = self.tokenizer(prompt, return_tensors="pt").to("cuda")

        streamer = TextIteratorStreamer(self.tokenizer, skip_special_tokens=True)

        generation_kwargs = dict(
            **inputs,
            max_new_tokens=max_tokens,
            streamer=streamer,
        )

        thread = Thread(target=self.model.generate, kwargs=generation_kwargs)
        thread.start()

        for token in streamer:
            yield token

        thread.join()


@app.function(image=image)
@modal.asgi_app()
def api():
    from fastapi import FastAPI
    from sse_starlette.sse import EventSourceResponse
    from pydantic import BaseModel

    web_app = FastAPI()

    class GenerateRequest(BaseModel):
        prompt: str
        max_tokens: int = 512

    @web_app.post("/generate-stream")
    async def generate_stream_endpoint(req: GenerateRequest):
        llm = LLMServer()

        async def event_generator():
            for token in llm.generate_stream.remote_gen(req.prompt, req.max_tokens):
                yield {"data": token}

        return EventSourceResponse(event_generator())

    return web_app

Next.js Streaming Handler

// app/api/chat/route.ts
import { NextRequest } from 'next/server';

const MODAL_API_URL = process.env.MODAL_LLM_URL!;

export async function POST(req: NextRequest) {
  const { prompt } = await req.json();

  const response = await fetch(`${MODAL_API_URL}/generate-stream`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
  });

  // Stream the response directly to client
  return new Response(response.body, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive',
    },
  });
}

React Component with Streaming

// components/Chat.tsx
'use client';

import { useState } from 'react';

export default function Chat() {
  const [prompt, setPrompt] = useState('');
  const [response, setResponse] = useState('');
  const [loading, setLoading] = useState(false);

  const sendMessage = async () => {
    setLoading(true);
    setResponse('');

    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
    });

    const reader = res.body?.getReader();
    const decoder = new TextDecoder();

    while (reader) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      // Parse SSE format
      const lines = chunk.split('\n');
      for (const line of lines) {
        if (line.startsWith('data: ')) {
          const token = line.slice(6);
          setResponse(prev => prev + token);
        }
      }
    }

    setLoading(false);
  };

  return (
    <div className="max-w-2xl mx-auto p-6">
      <textarea
        className="w-full p-4 border rounded-lg mb-4"
        value={prompt}
        => setPrompt(e.target.value)}
        placeholder="Ask something..."
      />

      <button
        className="bg-blue-600 text-white py-2 px-6 rounded-lg"
       
        disabled={loading}
      >
        {loading ? 'Generating...' : 'Send'}
      </button>

      {response && (
        <div className="mt-6 p-4 bg-gray-100 rounded-lg whitespace-pre-wrap">
          {response}
        </div>
      )}
    </div>
  );
}

Best Practices

1. Keep Modal API Keys Server-Side

// GOOD: API key in server-side API route
// app/api/process/route.ts
const MODAL_API_KEY = process.env.MODAL_API_KEY; // Server only

// BAD: Exposing API key to client
// NEXT_PUBLIC_MODAL_API_KEY=... // Never do this!

2. Use Warm Containers for Production

@app.cls(
    min_containers=1,           # Always keep 1 warm
    max_containers=10,          # Scale up to 10
    container_idle_timeout=300, # Keep warm for 5 min after use
)
class MyService:
    pass

3. Handle Vercel Timeouts

// Vercel timeout limits:
// - Hobby: 60 seconds
// - Pro: 300 seconds
// - Enterprise: 900 seconds

// For longer jobs, use webhooks instead of waiting

4. Implement Request Timeouts

const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 55000); // 55s

try {
  const response = await fetch(url, { signal: controller.signal });
} finally {
  clearTimeout(timeoutId);
}

5. Cache Modal Responses

// Next.js 15 cache pattern
import { unstable_cache } from 'next/cache';

const getCachedResult = unstable_cache(
  async (id: string) => {
    const response = await fetch(`${MODAL_API_URL}/process/${id}`);
    return response.json();
  },
  ['modal-result'],
  { revalidate: 3600 } // Cache for 1 hour
);

Common Pitfalls

1. CORS Errors

# Make sure CORS allows your Next.js domains
web_app.add_middleware(
    CORSMiddleware,
    allow_origins=[
        "http://localhost:3000",
        "https://*.vercel.app",  # All Vercel preview URLs
        "https://yourdomain.com",
    ],
    allow_methods=["*"],
    allow_headers=["*"],
)

2. Cold Start Latency

# Use min_containers=1 for production APIs
@app.function(min_containers=1)  # ~$1-2/day idle cost
def api():
    pass

3. Large Payloads

// For large files, upload directly to Modal or use signed URLs
// Don't pass >4MB through Next.js API routes

// Better: Use Modal's CloudBucketMount

4. Error Handling

// Always handle Modal errors gracefully
try {
  const response = await fetch(MODAL_API_URL);
  if (!response.ok) {
    // Log for debugging
    console.error(`Modal error: ${response.status}`, await response.text());
    // Return user-friendly error
    return NextResponse.json(
      { error: 'Service temporarily unavailable' },
      { status: 503 }
    );
  }
} catch (error) {
  // Handle network errors
  console.error('Network error:', error);
  return NextResponse.json(
    { error: 'Could not connect to service' },
    { status: 503 }
  );
}

Deployment Checklist

1. Deploy Modal backend first

   modal deploy modal_backend/app.py
   

2. Set environment variables in Vercel

   MODAL_API_URL=https://your-workspace--app-api.modal.run
   MODAL_API_KEY=your-secret-key
   WEBHOOK_SECRET=your-webhook-secret
   

3. Update CORS for production domain

4. Enable warm containers for production

   @app.function(min_containers=1)
   

5. Monitor costs

   modal app stats your-app-name
   

Related Skills

• nextjs-server-actions - Server Actions patterns
• nextjs-caching - Caching strategies
• nextjs-deployment - Deployment configurations