372 007 Supplemental Cloud Run Gemini Tuning Reference A8bd0e14

Supplemental Reference: Cloud Run & Gemini Fine-Tuning

Created: November 13, 2025 Purpose: Consolidated reference documentation for Cloud Run deployment and Gemini model fine-tuning Sources: Official Google Cloud documentation

Cloud Run Overview

What is Cloud Run?

Cloud Run is Google's fully managed platform for deploying and running containerized applications, functions, and code without managing infrastructure. Key principle:

"You don't have to create a cluster or manage infrastructure to be productive."

Core Philosophy

Serverless Containers:

• Deploy any containerized application
• No infrastructure management required
• Automatic scaling from zero to thousands of instances
• Pay only for actual usage
• Built-in security and compliance

Language Agnostic:

• Supports any programming language that can be containerized
• Source-based deployment for: Go, Node.js, Python, Java, .NET, Ruby
• Automatic container building following language best practices
• No container expertise required for supported languages

Three Execution Models

1. Services (Request-Driven)

Use Case: Respond to HTTP requests via stable HTTPS endpoints

Features:

• Unique HTTPS endpoints per service
• Automatic TLS certificate management
• WebSocket, HTTP/2, and gRPC support
• Automatic scaling based on incoming traffic
• Scale to zero when idle (no traffic = no cost)
• Traffic splitting for gradual rollouts

Example Scenarios:

• REST APIs
• Web applications
• Microservices
• ADK agents deployed with UI
• Firebase Cloud Functions alternative

Deployment:

gcloud run deploy SERVICE_NAME \
    --source . \
    --region us-central1 \
    --allow-unauthenticated

2. Jobs (Task-Driven)

Use Case: Execute parallelizable tasks to completion, optionally on a schedule

Features:

• Run once, or on a schedule (cron)
• Parallel task execution
• Automatic retries on failure
• No persistent endpoint (tasks finish and shut down)
• Cost-effective for batch processing

Example Scenarios:

• Data processing pipelines
• Scheduled ETL jobs
• Batch inference for ML models
• Database migrations
• Report generation

Deployment:

gcloud run jobs create JOB_NAME \
    --source . \
    --region us-central1 \
    --task-timeout 1h \
    --max-retries 3

3. Worker Pools (Event-Driven)

Use Case: Handle continuous pull-based workloads

Features:

• Long-running processes
• Pull from Pub/Sub, Kafka, queues
• Maintain persistent connections
• Automatic scaling based on queue depth
• Always-on instances (minimum instance count)

Example Scenarios:

• Kafka consumers
• Pub/Sub message processors
• Queue workers (Redis, RabbitMQ)
• Stream processing
• Real-time data ingestion

Deployment:

gcloud run workers create WORKER_NAME \
    --source . \
    --region us-central1 \
    --min-instances 1 \
    --max-instances 10

Key Service Features

1. Unique HTTPS Endpoints

Automatic Infrastructure:

• Each service gets a unique, stable HTTPS URL
• Automatic TLS certificate provisioning and renewal
• Global load balancing included
• Custom domain mapping support

Protocol Support:

• HTTP/1.1, HTTP/2, HTTP/3
• WebSocket connections
• gRPC (binary protocol)
• Server-Sent Events (SSE)

Example URL:

https://service-name-xyz123-uc.a.run.app

2. Auto-Scaling

Dynamic Scaling:

• Rapidly adjusts from zero to thousands of instances
• Based on incoming request volume
• CPU utilization
• Custom metrics (via Cloud Monitoring)

Scaling Configuration:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: my-service
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/minScale: "0"
        autoscaling.knative.dev/maxScale: "1000"
        autoscaling.knative.dev/target: "80"  # Target 80 concurrent requests per instance

Scale to Zero:

• No traffic = no running instances = no cost
• First request triggers cold start (~1-5 seconds)
• Keep warm with minimum instances if needed

3. Pay-Per-Use Pricing

Billing Model:

• Charge only for actual usage (not idle time)
• CPU and memory billed per 100ms increments
• Network egress charges apply
• Generous free tier:
  • 2 million requests/month
  • 360,000 vCPU-seconds/month
  • 180,000 GiB-seconds/month

Cost Optimization:

• Scale to zero when idle
• Right-size CPU and memory
• Use minimum instances sparingly
• Optimize cold start time

4. Disposable Containers

Stateless Design:

• Each instance has an in-memory filesystem
• Data does NOT persist across instances
• Instances can be replaced at any time
• No guaranteed execution order

External Storage Required:

• Use Cloud Storage for files
• Use Firestore/Cloud SQL for databases
• Use Memorystore for caching
• Use Pub/Sub for messaging

Best Practices:

# ❌ BAD: Store data in local filesystem
with open('/tmp/user_data.json', 'w') as f:
    json.dump(data, f)

# ✅ GOOD: Store data in Cloud Storage
from google.cloud import storage
client = storage.Client()
bucket = client.bucket('my-bucket')
blob = bucket.blob('user_data.json')
blob.upload_from_string(json.dumps(data))

5. Traffic Management

Gradual Rollouts:

• Deploy new revision without sending traffic
• Split traffic between revisions (e.g., 90% old, 10% new)
• Monitor metrics and gradually increase traffic
• Instant rollback if issues detected

Traffic Splitting Example:

# Deploy new revision without traffic
gcloud run deploy SERVICE_NAME --source . --no-traffic

# Split traffic: 90% to revision-1, 10% to revision-2
gcloud run services update-traffic SERVICE_NAME \
    --to-revisions revision-1=90,revision-2=10

Use Cases:

• Canary deployments
• A/B testing
• Blue-green deployments
• Feature flags

Integration Ecosystem

Cloud Run seamlessly integrates with Google Cloud services without managing connections:

Database Integration

Cloud SQL:

• Private VPC connection
• Unix socket connections
• Connection pooling built-in

Firestore:

• Native SDK support
• Application Default Credentials
• No explicit connection management

AlloyDB:

• High-performance PostgreSQL
• Private Service Connect

Storage Integration

Cloud Storage:

• Upload/download files
• Signed URLs for temporary access
• Object lifecycle management

Messaging & Events

Pub/Sub:

• Push subscriptions to Cloud Run
• Pull via Worker Pools
• Ordered message delivery

Eventarc:

• Event-driven architecture
• Trigger on Cloud Storage events
• Audit log triggers
• Custom events

AI/ML Integration

Vertex AI:

• ADK agent deployment target
• Gemini API calls
• Embeddings generation
• Model inference

Firebase ML:

• Custom model deployment
• AutoML integration

Monitoring & Logging

Cloud Logging:

• Automatic log ingestion
• Structured logging support
• Log-based metrics

Cloud Monitoring:

• Request latency metrics
• Error rates
• Custom metrics

ADK Deployment to Cloud Run

Single Command Deployment

adk deploy cloud_run \
    --project PROJECT_ID \
    --region us-central1 \
    --service_name weather-agent \
    --session_service_uri=agentengine://AGENT_ENGINE_ID \
    --memory_service_uri=agentengine://AGENT_ENGINE_ID \
    --app_name weather_agent \
    --with_ui \
    . \
    -- --allow-unauthenticated \
    --no-user-output-enabled

Key Parameters

ADK-Specific:

• --session_service_uri: Vertex AI Sessions service
• --memory_service_uri: Vertex AI Memory Bank
• --app_name: Agent application name
• --with_ui: Enable ADK Web UI for testing

Cloud Run-Specific:

• --allow-unauthenticated: Public access (testing only)
• --no-user-output-enabled: Disable user output logging

Production Configuration

For production deployments:

1. Require authentication: Replace --allow-unauthenticated with --no-allow-unauthenticated
2. Expose REST API or A2A endpoint instead of Web UI
3. Set resource limits: --memory 2Gi --cpu 2
4. Configure scaling: --min-instances 1 --max-instances 100
5. Enable VPC connector: --vpc-connector projects/PROJECT/locations/REGION/connectors/CONNECTOR

Gemini Model Supervised Fine-Tuning

What is Supervised Fine-Tuning?

Supervised fine-tuning (SFT) adapts Gemini models using labeled datasets to:

• Adjust model weights
• Minimize prediction differences from actual labels
• Specialize model behavior for domain-specific tasks

When to Use Fine-Tuning

Ideal Use Cases:

• Content differs significantly from model's original training data
• Prompts not producing expected results consistently
• Need for specific output formatting
• Domain-specific terminology and patterns
• Consistent behavioral patterns required

Not Recommended When:

• Few-shot prompting works well
• Simple formatting adjustments needed
• General-purpose responses sufficient

Supported Models

Current Fine-Tuning Support (November 2025)

Gemini 2.x Family:

• Gemini 2.5 Pro
• Gemini 2.5 Flash
• Gemini 2.5 Flash-Lite
• Gemini 2.0 Flash
• Gemini 2.0 Flash-Lite

Note: Gemini 1.5 models are NOT supported for fine-tuning.

Key Use Cases for Fine-Tuning

1. Classification

Task: Generate single-word or phrase responses

Example:

{
  "contents": [
    {"role": "user", "parts": [{"text": "Is this email spam? Subject: 'WINNER! Claim your prize now!'"}]},
    {"role": "model", "parts": [{"text": "spam"}]}
  ]
}

Use Cases:

• Content moderation
• Sentiment analysis
• Intent classification
• Entity recognition

2. Summarization

Task: Follow specific formats (e.g., remove PII, structured summaries)

Example:

{
  "contents": [
    {"role": "user", "parts": [{"text": "Summarize this patient record: John Smith, DOB 01/15/1980, diagnosed with hypertension..."}]},
    {"role": "model", "parts": [{"text": "Patient diagnosed with hypertension. Treatment plan initiated."}]}
  ]
}

Use Cases:

• Medical record summarization (HIPAA-compliant)
• Legal document summaries
• Customer interaction summaries

3. Extractive Question Answering

Task: Return exact substrings from source material

Example:

{
  "contents": [
    {"role": "user", "parts": [{"text": "What is the refund policy? Document: 'Refunds available within 30 days of purchase.'"}]},
    {"role": "model", "parts": [{"text": "Refunds available within 30 days of purchase."}]}
  ]
}

Use Cases:

• FAQ systems
• Document Q&A
• Contract analysis

4. Chat (Persona-Based)

Task: Implement specific personas or behavioral patterns

Example:

{
  "contents": [
    {"role": "user", "parts": [{"text": "What's the weather like?"}]},
    {"role": "model", "parts": [{"text": "Arrr, the weather be fine today, matey! Clear skies ahead!"}]}
  ]
}

Use Cases:

• Brand-specific chatbots
• Character-based interactions
• Industry-specific assistants

Technical Specifications

Dataset Limitations (Gemini 2.5 Flash/Flash-Lite)

Size Limits:

• Maximum training tokens per example: 131,072 (128K context)
• Maximum validation dataset: 5,000 examples
• Maximum file size: 1GB (JSONL format)
• Maximum dataset size:
  • Text-only: 1,000,000 examples
  • Multimodal: 300,000 examples

Format Requirements:

• JSONL (JSON Lines) format
• One example per line
• Each example must follow the schema

Adapter Sizes

Gemini 2.5 Flash/Flash-Lite:

• Available sizes: 1, 2, 4, 8, 16
• Larger adapters = more capacity but slower training
• Smaller adapters = faster training but less adaptation

Gemini 2.5 Pro:

• Available sizes: 1, 2, 4, 8
• Same trade-offs as Flash

Recommendation:

• Start with adapter size 4
• Increase if underfitting
• Decrease if overfitting or slow training

Training Data Format

Required Schema:

{
  "contents": [
    {
      "role": "user",
      "parts": [{"text": "User prompt here"}]
    },
    {
      "role": "model",
      "parts": [{"text": "Expected model response"}]
    }
  ]
}

Multi-Turn Conversations:

{
  "contents": [
    {"role": "user", "parts": [{"text": "First user message"}]},
    {"role": "model", "parts": [{"text": "First model response"}]},
    {"role": "user", "parts": [{"text": "Follow-up question"}]},
    {"role": "model", "parts": [{"text": "Follow-up response"}]}
  ]
}

Multimodal Examples:

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {"text": "What's in this image?"},
        {"inline_data": {"mime_type": "image/jpeg", "data": "base64_encoded_image"}}
      ]
    },
    {
      "role": "model",
      "parts": [{"text": "A cat sitting on a couch"}]
    }
  ]
}

Fine-Tuning Workflow

1. Prepare Training Data

import json

training_examples = []

for data_point in your_dataset:
    example = {
        "contents": [
            {"role": "user", "parts": [{"text": data_point['prompt']}]},
            {"role": "model", "parts": [{"text": data_point['completion']}]}
        ]
    }
    training_examples.append(example)

# Save as JSONL
with open('train_data.jsonl', 'w') as f:
    for example in training_examples:
        f.write(json.dumps(example) + '\n')

2. Upload to Cloud Storage

gsutil cp train_data.jsonl gs://YOUR_BUCKET/tuning_data/
gsutil cp validation_data.jsonl gs://YOUR_BUCKET/tuning_data/

3. Launch Fine-Tuning Job

from google.genai import Client as VertexClient
from google.genai import types as genai_types

# Initialize Vertex AI client
vertex_client = VertexClient(vertexai=True, project=PROJECT_ID, location=REGION)

# Configure training
training_dataset = {"gcs_uri": "gs://YOUR_BUCKET/tuning_data/train_data.jsonl"}
validation_dataset = genai_types.TuningValidationDataset(
    gcs_uri="gs://YOUR_BUCKET/tuning_data/validation_data.jsonl"
)

# Start tuning job
tuning_job = vertex_client.tunings.tune(
    base_model="gemini-2.5-flash",
    training_dataset=training_dataset,
    config=genai_types.CreateTuningJobConfig(
        adapter_size="ADAPTER_SIZE_FOUR",
        epoch_count=3,
        tuned_model_display_name="my-tuned-model",
        validation_dataset=validation_dataset,
    ),
)

print(f"Tuning job created: {tuning_job.name}")

4. Monitor Job Progress

import time

# Poll job status
while tuning_job.state in [
    genai_types.JobState.JOB_STATE_PENDING,
    genai_types.JobState.JOB_STATE_RUNNING,
]:
    print(f"Status: {tuning_job.state}")
    time.sleep(60)  # Check every minute
    tuning_job = vertex_client.tunings.get(name=tuning_job.name)

print(f"Final status: {tuning_job.state}")

if tuning_job.state == genai_types.JobState.JOB_STATE_SUCCEEDED:
    print(f"Tuned model endpoint: {tuning_job.tuned_model.endpoint}")

5. Evaluate Tuned Model

# Query tuned model
response = vertex_client.models.generate_content(
    model=tuning_job.tuned_model.endpoint,
    contents=[{"role": "user", "parts": [{"text": "Test prompt"}]}],
    config={"temperature": 0.1, "max_output_tokens": 50},
)

print(response.text)

Evaluation and Deployment

Integrated Evaluation

Gen AI Evaluation Service:

• Automatic evaluation for supported models and regions
• Track quality metrics over time
• Compare base model vs. tuned model performance

Evaluation Metrics:

• Accuracy
• Precision/Recall (classification tasks)
• ROUGE scores (summarization)
• BLEU scores (translation)
• Custom metrics

Deployment Options

Inference Pricing:

• Tuned models use same pricing as base models
• No additional inference cost
• Pay for tuning job compute time only

Deployment Methods:

1. Direct API calls via Vertex AI SDK
2. Cloud Run deployment for scalable serving
3. Agent Engine integration for agent-based apps

Important Constraints

⚠️ No Controlled Generation at Inference:

"Models aren't trained to handle controlled generation during tuning."

What This Means:

• Don't use JSON mode with tuned models
• Don't apply strict output constraints at inference
• Train the model to produce desired format instead

Best Practices

Data Preparation

1. Quality over Quantity:

   • 100 high-quality examples > 1,000 low-quality examples
   • Ensure consistency in formatting
   • Remove noisy or contradictory examples

2. Balanced Dataset:

   • Equal representation of classes (classification)
   • Diverse examples covering edge cases
   • Validation set should match test distribution

3. Format Consistency:

   • Consistent prompt structure
   • Consistent completion format
   • Follow JSONL schema exactly

Training Configuration

1. Start Small:

   • Begin with adapter size 4
   • Use 3 epochs initially
   • Evaluate before scaling up

2. Monitor Validation Loss:

   • Track validation metrics during training
   • Stop early if overfitting detected
   • Use validation dataset for tuning decisions

3. Regional Configuration:

   • Choose region close to data source
   • Ensure data residency compliance
   • Consider latency requirements

Evaluation Strategy

1. Hold-Out Test Set:

   • Never use test data for training or validation
   • Evaluate final model on unseen data
   • Report metrics on test set only

2. Qualitative Review:

   • Manually inspect model outputs
   • Check for edge cases and failure modes
   • Validate against business requirements

3. A/B Testing:

   • Compare tuned model vs. base model in production
   • Measure business metrics (conversion, satisfaction)
   • Roll back if tuned model underperforms

Related jeremy-* Plugins

jeremy-vertex-engine

• Vertex AI Agent Engine deployment and management
• Runtime configuration validation
• Production readiness checks

jeremy-vertex-validator

• Validate fine-tuning datasets
• Check JSONL format compliance
• Production deployment validation

jeremy-genkit-pro

• Firebase Genkit integration with tuned Gemini models
• Cloud Run deployment automation
• RAG with fine-tuned models

jeremy-vertex-terraform

• Terraform infrastructure for tuning jobs
• Automated pipeline deployment
• Resource provisioning

jeremy-adk-orchestrator

• ADK agents with fine-tuned Gemini models
• Multi-agent systems on Cloud Run
• Memory Bank integration

Quick Reference

Cloud Run Deployment

gcloud run deploy SERVICE_NAME --source . --region us-central1

ADK to Cloud Run

adk deploy cloud_run --project PROJECT_ID --region REGION --service_name SERVICE

Start Fine-Tuning Job

tuning_job = vertex_client.tunings.tune(
    base_model="gemini-2.5-flash",
    training_dataset={"gcs_uri": "gs://bucket/train.jsonl"},
    config={"adapter_size": "ADAPTER_SIZE_FOUR", "epoch_count": 3}
)

Check Tuning Job Status

job = vertex_client.tunings.get(name=tuning_job.name)
print(job.state)

Query Tuned Model

response = vertex_client.models.generate_content(
    model=tuned_model_endpoint,
    contents=[{"role": "user", "parts": [{"text": "prompt"}]}]
)

Documentation Version: November 2025 Last Updated: 2025-11-13 Status: Production-Ready Reference