Gemini API Dev

Gemini Api Dev

When to Use

Trigger phrases:

• "gemini api dev"

• "When building apps powered by Google Gemini models"

• "When processing multimodal inputs (text + images + audio + video)"

• "When implementing function calling or tool use with Gemini"

• When building apps powered by Google Gemini models

• When processing multimodal inputs (text + images + audio + video)

• When implementing function calling or tool use with Gemini

• When using Gemini for grounding with Google Search

When NOT to Use

• For OpenAI-compatible APIs (use OpenAI skills)
• For local model inference (use Ollama skills)
• For fine-tuning (use Vertex AI training skills)

Overview

The Google Gemini API provides access to Google's most capable AI models — Gemini 2.5 Pro, 2.5 Flash, and Gemini Nano — through a unified SDK. It supports text generation, multimodal understanding (images, audio, video, PDFs), function calling, and grounding with Google Search. The API is available via the google-genai Python SDK and @google/genai TypeScript SDK, both providing first-class async support, streaming generators, and type-safe response handling.

Gemini 2.5 Pro excels at complex reasoning, code generation, and multi-turn conversations with a 1-million-token context window. Gemini 2.5 Flash is optimized for speed and cost-efficiency while maintaining strong reasoning capabilities, making it suitable for high-volume production workloads. Nano models run on-device for Android and Chrome, handling summarization, smart reply, and text classification without network calls.

The API architecture follows a content-based message model: each request sends contents arrays with role (user or model) and parts — which can be text, inline_data (base64-encoded images/audio), file_data (PDFs, videos), function_call, or function_response. Configuration options include systemInstruction for persona setting, temperature (0.0–2.0), topP, topK, maxOutputTokens, stopSequences, and safetySettings for content filtering.

Grounding with Google Search enables responses that reference real-time information, making Gemini suitable for news analysis, fact-checking, and research tasks. Enabled via googleSearch grounding in the generation config, it returns groundingMetadata with source URLs, supporting citations alongside generated text. This feature is especially valuable for applications requiring current, verifiable information.

Streaming is a first-class pattern across both SDKs — generateContentStream returns async generators that yield chunks as they become available, enabling responsive UI updates, early processing of partial results, and progressive rendering of long outputs without buffering.

Workflow

1. Set up SDK — Install google-genai (Python) or @google/genai (TypeScript). Configure API key from Google AI Studio or service account for Vertex AI. Load from GEMINI_API_KEY environment variable.
2. Configure auth — API key for Google AI Studio, service account for Vertex AI. For production, use service account with workload identity federation to avoid hardcoded keys.
3. Define system instruction — Set model persona, output format constraints, tone, and domain expertise via systemInstruction in generation config. System instructions are treated as developer messages and significantly improve output consistency.
4. Prepare contents — Build message history with alternating user/model roles. Include multimodal parts: inline_data for base64 images/audio, file_data for PDFs/videos uploaded via File API, or text blocks. Use role: "user" for input and role: "model" for prior assistant turns.
5. Configure generation parameters — Set temperature (0.0–2.0), top_p, top_k, max_output_tokens, stop_sequences, and safety_settings per request. For deterministic output, set a seed parameter (available in Pro models).
6. Handle responses — Stream via generateContentStream async generator for real-time output. Handle function_call parts by executing tools and resuming with function_response parts. Process grounded responses that include groundingMetadata with source citations and groundingChunks.
7. Error handling — Catch ClientError (4xx) and ServerError (5xx). Implement exponential backoff with jitter for 429 RESOURCE_EXHAUSTED rate limits. Fallback between Pro and Flash models on quota exhaustion per region.

Anti-Rationalization Table

Code Example (TypeScript)

import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContentStream({
  model: 'gemini-2.5-pro',
  contents: [{ role: 'user', parts: [{ text: 'Analyze this chart' }] }],
  config: {
    systemInstruction: 'You are a data analyst. Be precise and cite numbers.',
    temperature: 0.3,
  }
});

for await (const chunk of response) {
  process.stdout.write(chunk.text);
}

Code Example (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

# Basic text generation
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Explain how RAG works in two sentences.",
)
print(response.text)

# Multimodal input with image
import httpx
img = httpx.get("https://example.com/chart.png").content
response = client.models.generate_content(
    model="gemini-2.5-pro",
    contents=[
        "Analyze this chart and summarize the trend.",
        types.Part.from_bytes(data=img, mime_type="image/png"),
    ],
)
print(response.text)

# Streaming
stream = client.models.generate_content_stream(
    model="gemini-2.5-pro",
    contents="Write a short story about AI.",
    config=types.GenerateContentConfig(
        temperature=0.7,
        max_output_tokens=2048,
    ),
)
for chunk in stream:
    print(chunk.text, end="")

# Function calling
def get_weather(city: str) -> str:
    return f"Weather in {city}: sunny, 25\u00b0C"

tools = [
    types.Tool(function_declarations=[
        types.FunctionDeclaration(
            name="get_weather",
            description="Get current weather for a city",
            parameters={
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"}
                },
                "required": ["city"],
            },
        ),
    ]),
]

response = client.models.generate_content(
    model="gemini-2.5-pro",
    contents="What's the weather in Tokyo?",
    config=types.GenerateContentConfig(tools=tools),
)

if response.candidates[0].content.parts[0].function_call:
    fc = response.candidates[0].content.parts[0].function_call
    result = get_weather(fc.args["city"])
    # Send function response back to continue conversation
    response = client.models.generate_content(
        model="gemini-2.5-pro",
        contents=[
            "What's the weather in Tokyo?",
            types.Part.from_function_call(fc),
            types.Part.from_function_response(
                name=fc.name,
                response={"result": result},
            ),
        ],
        config=types.GenerateContentConfig(tools=tools),
    )
    print(response.text)

Setup / Configuration

Installation

Python:

pip install google-genai

TypeScript / Node.js:

npm install @google/genai

Authentication

1. Go to Google AI Studio and click Get API Key
2. Create an API key (free tier includes 60 requests per minute)
3. Set it as environment variable:

   export GEMINI_API_KEY="your-api-key-here"
   

For production deployments, use a Vertex AI service account with workload identity federation to avoid embedding API keys in application code.

Safety Settings

Gemini safety filters operate on four categories: harassment, hate speech, sexually explicit content, and dangerous content. Each has adjustable thresholds:

Override via generation config:

config = types.GenerateContentConfig(
    safety_settings=[
        types.SafetySetting(
            category="HARM_CATEGORY_HARASSMENT",
            threshold="BLOCK_ONLY_HIGH",
        ),
    ],
)

Model Selection Guide

Common Issues / Troubleshooting

Monetization

Process

1. Design — Define interface, identify patterns, plan implementation
2. Implement — Write code following existing conventions, add tests
3. Verify — Run tests, check integration, validate behavior

Verification

• API key authenticated successfully via Google AI Studio
• Chat completions return valid responses for multiple models (Pro, Flash)
• Streaming works without buffering issues or missing chunks
• Function calling triggers correctly and resumes conversation
• Safety settings block harmful content at configured thresholds
• Multimodal input (image/audio/PDF) processed without errors
• Grounding with Google Search returns citations in response
• Error handling catches rate limit (429) and permission (403) errors
• System instruction correctly influences model behavior
• Token usage tracked and within quota limits