＋ Add to collection

Or: paste this in your AI chat

Please install the Agent Skill "google/agent-platform-inference" for me from skillmd.com. A SkillMD skill is a SKILL.md file — plain markdown plus a few companion files — that you save into my skills directory. Use whichever fetch route fits your setup: (1) if you have a terminal, run npx -y skillmds add google/agent-platform-inference; (2) otherwise fetch the file manifest at https://api.skillmd.com/api/skills/google/agent-platform-inference/bundle?format=manifest and download each listed file from its "url" into agent-platform-inference/ in your skills directory; (3) if the SkillMD MCP tools are connected, call skillmd_download with slug "google/agent-platform-inference" and save the returned files under agent-platform-inference/. Show me the SKILL.md before saving if you'd like me to review it. If your skills directory is read-only (e.g. the Claude desktop app), save the files to a working folder instead and tell me — I'll add them through my app's skills settings.

README badge

Show this skill's registry status in your repo:

[![SkillMD](https://skillmd.com/badge/google/agent-platform-inference.svg)](https://skillmd.com/skills/google/agent-platform-inference)

Agent Platform Inference

Name: Agent Platform Inference
Rating: 4.2 (5 reviews)
Author: Google

by Google google/agent-platform-inference multi-file Updated Jul 6, 2026

Authenticates and connects to Google Cloud Agent Platform for inference with Gemini and third-party OpenMaaS models (Llama, DeepSeek, Qwen). Generates code for multiple SDKs, configures endpoints, and troubleshoots common errors.

SKILL.md

Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google Cloud Agent Platform to use Generative AI models. It covers both First-Party (Gemini) and Third-Party (OpenMaaS) models.

Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere to the following safety tiers based on the action requested. (The skill is read-only; other safety tiers are omitted):

Tier R: Read-only / Inference (client.models.generate_content, client.chat.completions.create, client.completions.create, client.embeddings.create)
- Requires interactive confirmation with 'Yes'/ 'No' options before executing model inference on behalf of the user, to prevent unexpected cost or quota consumption. The confirmation prompt must clearly explain the proposed inference execution and its key parameters (e.g., target model ID, SDK choice, input prompt). Natural-language paraphrases without specifying the parameters are NOT sufficient.
- Same-turn restriction: Do not execute the inference scripts or commands in the same turn as presenting the confirmation prompt. Stop and wait for the user's reply; only execute after explicit 'Yes' / approval.
- Gold Standard Example:
  I will perform model inference with the following parameters. Please confirm this information before I proceed:
  - Model ID: deepseek-ai/deepseek-v3.2-maas
  - SDK: OpenAI SDK (via Vertex AI Endpoint)
  - Input Prompt: "Explain the concept of quantum computing..." Do you confirm? [Yes/No]

Phase 0: Environment Setup

CRITICAL: Before running any of the Python sample scripts in the scripts/ directory (e.g., scripts/openmaas_openai_sdk.py), you MUST ensure the environment is correctly initialized by following these steps:

Google Cloud Authentication: Authenticate with your Google Cloud credentials and configure active Application Default Credentials (ADC) for Agent Platform access:
```
gcloud auth login
gcloud auth application-default login
```

Enable API (if not already enabled):

gcloud services enable aiplatform.googleapis.com

Virtual Environment: Create and activate a dedicated local virtual environment:
```
python3 -m venv .venv
source .venv/bin/activate
```
Install Dependencies: Install the required SDKs:
```
pip install -r scripts/requirements.txt
```
Verify Setup (Optional): Run all sample scripts at once to verify the environment is working end-to-end:
```
./scripts/verify_all.sh
```
Execution: Advise the user that every time they execute a Python snippet from this skill, they must ensure this virtual environment is activated first.

[!IMPORTANT] CRITICAL: Model IDs & Availability

Gemini Models: See Gemini Models for valid Model IDs and Regions.

OpenMaaS Models: See [Use Open Models on Agent Platform] (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models) for Llama, DeepSeek, Qwen, etc.

Incomplete Lists: The Model IDs listed in this skill are examples only and may be incomplete or outdated.

Action: Always verify the Model ID and Region using the links above before generating code.

Workflow Decision Tree

Model Family Identification: Has the user specified whether they want to call a Gemini (First-Party) model or an OpenMaaS (Third-Party, e.g. Llama, DeepSeek, Qwen) model?
- No -> Ask the user which model family they want to use. If they provide a specific model name, infer the family from the name.
- Yes -> Proceed to Step 2.
SDK Choice: Which SDK does the user want to use?
- Gemini + GenAI SDK (preferred for Gemini) -> Proceed to [1. Gemini Models].
- Gemini + legacy Vertex AI SDK -> Proceed to [1. Gemini Models].
- OpenMaaS + OpenAI SDK (preferred for OpenMaaS) -> Proceed to [2. OpenMaaS Models].
- OpenMaaS + GenAI SDK -> Proceed to [2. OpenMaaS Models].
- Unsure -> Default to the preferred SDK for the chosen family.
Troubleshooting: Is the user reporting an error (429 Resource Exhausted, 400 User Validation, 404 Not Found, etc.)?
- Yes -> Proceed to [3. Troubleshooting & Common Error Codes].
- No -> Proceed with the SDK choice from Step 2.

1. Gemini Models

For Gemini models (e.g., gemini-2.5-pro, gemini-3-flash-preview), the GenAI SDK (google-genai) is the PREFERRED method. The legacy vertexai SDK is still supported but GenAI SDK is recommended for new projects.

[!IMPORTANT] Preview Models (including Gemini 3.1) are often ONLY available in the global region. Stable models are available in us-central1 and other regions.

Choosing the Right SDK

Gemini Models: GenAI SDK (google-genai) is PREFERRED. Use OpenAI SDK for compatibility, or Legacy SDK (vertexai) if needed.
OpenMaaS Models: OpenAI SDK is HIGHLY RECOMMENDED. Use GenAI SDK or Legacy SDK if you have specific infrastructure requirements.

Installation

pip install google-genai

Python Example (GenAI SDK - Preferred)

See scripts/gemini_genai_sdk.py for the complete code.

Alternative: OpenAI SDK (Chat Completions)

Use the standard OpenAI SDK with the Agent Platform endpoint. This is great for cross-compatibility.

See scripts/gemini_openai_sdk.py for the complete code.

Legacy: Agent Platform SDK

The legacy vertexai SDK is still widely used but google-genai is preferred for new Gemini projects.

See scripts/gemini_vertexai_sdk.py for the complete code.

Documentation: Google GenAI SDK

Documentation: Agent Platform Gemini Models

2. OpenMaaS Models (Llama, DeepSeek, Qwen, etc.)

For OpenMaaS (Model-as-a-Service) models, the HIGHLY RECOMMENDED approach is to use the standard OpenAI SDK with a specific Vertex AI endpoint.

[!WARNING] While GenerativeModel can support some OpenMaaS models, it is discouraged. Use the OpenAI SDK for best compatibility (especially for Chat Completions).

Installation

pip install openai google-auth

Authentication for OpenAI SDK

You MUST use a Google Cloud OAuth access token as the API key for the OpenAI SDK.

import google.auth
from google.auth.transport.requests import Request

def get_gcp_access_token():
    creds, _ = google.auth.default()
    creds.refresh(Request())
    return creds.token

[!NOTE] Google Cloud access tokens typically expire after 1 hour. The get_gcp_access_token() function above retrieves a fresh token at the time it is called.

For long-running applications, you implement a refresh mechanism. See Refresh the access token for details.

Configuration (Base URL)

Global Endpoint (Recommended for most models requiring global availability): https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/global/endpoints/openapi
Regional Endpoint: https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/openapi

Python Example (OpenMaaS - Chat Completions)

See scripts/openmaas_openai_sdk.py for the complete code.

[!TIP] Alternative: Environment Variables You can set environment variables in your shell instead of updating the code.
export OPENAI_BASE_URL="https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT_ID/locations/global/endpoints/openapi"
export OPENAI_API_KEY="$(gcloud auth print-access-token)"
Then initialize the client without arguments: client = OpenAI()

Python Example (OpenMaaS - Completions API)

The following models support the legacy Completions API: zai-org/glm-5-maas, moonshotai/kimi-k2-thinking-maas, minimaxai/minimax-m2-maas, deepseek-ai/deepseek-v3.1-maas, and deepseek-ai/deepseek-v3.2-maas.

response = client.completions.create(
    model="deepseek-ai/deepseek-v3.2-maas",
    prompt="Once upon a time",
    max_tokens=100
)
print(response.choices[0].text)

Python Example (OpenMaaS - Embeddings)

# Verify specific Embedding Model ID on Model Garden (e.g., intfloat/multilingual-e5-small)
response = client.embeddings.create(
    model="intfloat/multilingual-e5-large-maas",
    input="The quick brown fox jumps over the lazy dog",
)
print(response.data[0].embedding)

Alternative: GenAI SDK

The google-genai SDK can also access OpenMaaS models via the vertexai backend.

See scripts/openmaas_genai_sdk.py for the complete code.

[!IMPORTANT] Model ID Format: For GenAI SDK with OpenMaaS, you MUST use the full path: publishers/PUBLISHER/models/MODEL (e.g., publishers/zai-org/models/glm-5-maas).

Legacy: Agent Platform SDK (OpenMaaS)

For OpenMaaS, you can also use GenerativeModel (if supported).

See scripts/openmaas_vertexai_sdk.py for the complete code.

[!IMPORTANT] Model ID Format: For Agent Platform SDK with OpenMaaS, you MUST use the full path: publishers/PUBLISHER/models/MODEL.

Model Reference & Availability

Documentation: Use Open Models on Agent Platform

[!TIP] Self-Deployment for Control: If you need dedicated hardware (GPUs/TPUs), guaranteed capacity, or specific regional placement not offered by MaaS, you can Self-Deploy these models to Agent Platform Endpoints. Search for the model in Model Garden and click "Deploy" to select your machine type.

[!IMPORTANT] Finding Inference Examples: The list above is a starting point. For the definitive inference snippets (especially for Chat Completions payload structure):

Consult the Use Open Models on Agent Platform list.

Click the link for your specific model (e.g., "DeepSeek-V3") to visit its Model Garden page.

Look for the "Sample Code" or "Use this model" button on the Model Garden page to get the exact curl or Python code for that specific model version.

[!NOTE] This list is INCOMPLETE. See [Use Open Models on Agent Platform] (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models) for the full list of supported models.

Model Family	Model ID Examples	Location	Notes
Llama 4	`meta/llama-4-maverick-17b-128e-instruct-maas`	`us-east5`
Llama 4	`meta/llama-4-scout-17b-16e-instruct-maas`	`us-east5`
Llama 3.3	`meta/llama-3.3-70b-instruct-maas`	`us-central1`
DeepSeek	`deepseek-ai/deepseek-v3.2-maas`	`global`	Global ONLY
DeepSeek	`deepseek-ai/deepseek-v3.1-maas`	`us-west2`	US-West2 ONLY
DeepSeek	`deepseek-ai/deepseek-r1-0528-maas`	`us-central1`
Qwen 3	`qwen/qwen3-coder-480b-a35b-instruct-maas`	`global`
Qwen 3	`qwen/qwen3-next-80b-a3b-instruct-maas`	`global`
Kimi	`moonshotai/kimi-k2-thinking-maas`	`global`
MiniMax	`minimaxai/minimax-m2-maas`	`global`
GLM	`zai-org/glm-4.7-maas`, `zai-org/glm-5-maas`	`global`

3. Troubleshooting & Common Error Codes

429: Resource Exhausted

Cause: OpenMaaS and Gemini models use Dynamic Shared Quota (DSQ). Resources are pooled and allocated dynamically based on availability. A 429 error indicates the shared pool is temporarily exhausted, not necessarily that your specific project quota is hit (though it can be).
Solution: Implement strict exponential backoff and retry strategies.
High Throughput: For production workloads requiring high throughput or guaranteed capacity, consider Provisioned Throughput (PT).
Important: Quota increases through normal cloud processes (Cloud Console) are NOT applicable for DSQ constraints.
Documentation: Quotas and limits (DSQ)

400: User Validation Error

Cause: Invalid request format, unsupported parameter, or incorrect Model ID.
Action: Double-check your request payload and parameters. Verify the Model ID and Region are correct.

404: Not Found / Model Not Available

Cause: The model is not enabled, or not available in the specified project or region.
Action:
1. Check Location Availability:
  - OpenMaaS: Verify the model is available in your region. See Model Availability by Location.
  - Gemini:
    - Source of Truth: Always check Gemini Model Locations for the authoritative list.
    - Preview Models: All Preview models (e.g., Gemini 3.1, experimental versions) are often ONLY available in the us-central1 or global regions.
    - Stable Models: (e.g., Gemini 2.5 Pro) Available in us-central1, europe-west4, and many other regions.
    - Important: If you get a 404/400 error, try switching your client location to us-central1 or global.
2. Enable Llama Models: For Llama 3.3 and Llama 4, you MUST enable the model in Model Garden before use. Go to the [Model Garden] (https://console.cloud.google.com/agent-platform/model-garden), search for the model card (e.g., "Llama 3.3 API Service"), and click Enable. Only then can you make inference requests.

---
name: agent-platform-inference
description: Authenticates and connects to Google Cloud Agent Platform for inference with Gemini and third-party OpenMaaS models (Llama, DeepSeek, Qwen). Generates code for multiple SDKs, configures endpoints, and troubleshoots common errors.
---

# Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google
Cloud Agent Platform to use Generative AI models. It covers both First-Party
(Gemini) and Third-Party (OpenMaaS) models.

## Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere
to the following safety tiers based on the action requested. (The skill is
read-only; other safety tiers are omitted):

1.  **Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)**
    *   Requires **interactive confirmation** with 'Yes'/
    'No' options before executing model inference on
    behalf of the user, to prevent unexpected cost or
    quota consumption. The confirmation prompt must
    clearly explain the proposed inference execution and
    its key parameters (e.g., target model ID, SDK
    choice, input prompt). Natural-language paraphrases
    without specifying the parameters are NOT sufficient.
    *   **Same-turn restriction**: Do not execute the
    inference scripts or commands in the same turn as
    presenting the confirmation prompt. Stop and wait
    for the user's reply; only execute after explicit
    'Yes' / approval.
    *   **Gold Standard Example**:
        > I will perform model inference with the following parameters. Please
        > confirm this information before I proceed:
        > *   **Model ID**: `deepseek-ai/deepseek-v3.2-maas`
        > *   **SDK**: OpenAI SDK (via Vertex AI Endpoint)
        > *   **Input Prompt**: "Explain the concept of quantum computing..."
        > Do you confirm? [Yes/No]

## Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/`
directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the
environment is correctly initialized by following these steps:

1.  **Google Cloud Authentication**: Authenticate with your Google Cloud
    credentials and configure active Application Default Credentials (ADC) for
    Agent Platform access:
    
    ```bash
    gcloud auth login
    gcloud auth application-default login
    ```
2.  **Enable API** (if not already enabled):
    
    ```bash
    gcloud services enable aiplatform.googleapis.com
    ```
3.  **Virtual Environment**: Create and activate a dedicated local virtual
    environment:
    
    ```bash
    python3 -m venv .venv
    source .venv/bin/activate
    ```
4.  **Install Dependencies**: Install the required SDKs:
    
    ```bash
    pip install -r scripts/requirements.txt
    ```
5.  **Verify Setup (Optional)**: Run all sample scripts at once to verify the
    environment is working end-to-end:
    
    ```bash
    ./scripts/verify_all.sh
    ```
6.  **Execution**: Advise the user that every time they execute a Python
    snippet from this skill, they must ensure this virtual environment is
    activated first.

<!-- disableFinding(LINE_OVER_80) -->

> [!IMPORTANT]
> **CRITICAL: Model IDs & Availability**
> *   **Gemini Models**: See [Gemini Models][gemini-models-docs] for valid
>     Model IDs and Regions.
> *   **OpenMaaS Models**: See [Use Open Models on Agent Platform]
>     (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
>     for Llama, DeepSeek, Qwen, etc.
> *   **Incomplete Lists**: The Model IDs listed in this skill are **examples
>     only** and may be incomplete or outdated.
> *   **Action**: Always verify the Model ID and Region using the links above
>     before generating code.
>
> [gemini-models-docs]: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate
<!-- enableFinding(LINE_OVER_80) -->
## Workflow Decision Tree

1.  **Model Family Identification**: Has the user specified whether they want
    to call a **Gemini** (First-Party) model or an **OpenMaaS** (Third-Party,
    e.g. Llama, DeepSeek, Qwen) model?

    *   **No** -> Ask the user which model family they want to use. If they
        provide a specific model name, infer the family from the name.
    *   **Yes** -> Proceed to Step 2.

2.  **SDK Choice**: Which SDK does the user want to use?

    *   **Gemini + GenAI SDK** (preferred for Gemini) -> Proceed to
        [1. Gemini Models].
    *   **Gemini + legacy Vertex AI SDK** -> Proceed to [1. Gemini Models].
    *   **OpenMaaS + OpenAI SDK** (preferred for OpenMaaS) -> Proceed to
        [2. OpenMaaS Models].
    *   **OpenMaaS + GenAI SDK** -> Proceed to [2. OpenMaaS Models].
    *   **Unsure** -> Default to the preferred SDK for the chosen family.

3.  **Troubleshooting**: Is the user reporting an error (429 Resource
    Exhausted, 400 User Validation, 404 Not Found, etc.)?

    *   **Yes** -> Proceed to [3. Troubleshooting & Common Error Codes].
    *   **No** -> Proceed with the SDK choice from Step 2.

## 1. Gemini Models

For Gemini models (e.g., `gemini-2.5-pro`, `gemini-3-flash-preview`), the
**GenAI SDK** (`google-genai`) is the **PREFERRED** method. The legacy
`vertexai` SDK is still supported but GenAI SDK is recommended for new projects.

> [!IMPORTANT]
> **Preview Models (including Gemini 3.1)** are often **ONLY** available in the
> `global` region. Stable models are available in `us-central1` and other
> regions.

### Choosing the Right SDK

*   **Gemini Models**: **GenAI SDK** (`google-genai`) is **PREFERRED**. Use OpenAI SDK for compatibility, or Legacy SDK (`vertexai`) if needed.
*   **OpenMaaS Models**: **OpenAI SDK** is **HIGHLY RECOMMENDED**. Use GenAI SDK or Legacy SDK if you have specific infrastructure requirements.

### Installation

```bash
pip install google-genai
```

### Python Example (GenAI SDK - Preferred)

See [`scripts/gemini_genai_sdk.py`](scripts/gemini_genai_sdk.py) for the
complete code.

### Alternative: OpenAI SDK (Chat Completions)

Use the standard OpenAI SDK with the Agent Platform endpoint. This is great for
cross-compatibility.

See [`scripts/gemini_openai_sdk.py`](scripts/gemini_openai_sdk.py) for the
complete code.

### Legacy: Agent Platform SDK

The legacy `vertexai` SDK is still widely used but `google-genai` is preferred
for new Gemini projects.

See [`scripts/gemini_vertexai_sdk.py`](scripts/gemini_vertexai_sdk.py) for the
complete code.

**Documentation**: [Google GenAI SDK](https://github.com/googleapis/python-genai)

**Documentation**: [Agent Platform Gemini Models](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/google-models)

## 2. OpenMaaS Models (Llama, DeepSeek, Qwen, etc.)

For OpenMaaS (Model-as-a-Service) models, the **HIGHLY RECOMMENDED** approach is
to use the standard **OpenAI SDK** with a specific Vertex AI endpoint.

> [!WARNING]
> While `GenerativeModel` *can* support some OpenMaaS models, it is
**discouraged**. Use the OpenAI SDK for best compatibility (especially for Chat
Completions).

### Installation

```bash
pip install openai google-auth
```

### Authentication for OpenAI SDK

You **MUST** use a Google Cloud OAuth access token as the API key for the OpenAI
SDK.

```python
import google.auth
from google.auth.transport.requests import Request

def get_gcp_access_token():
    creds, _ = google.auth.default()
    creds.refresh(Request())
    return creds.token
```

> [!NOTE]
> Google Cloud access tokens typically expire after 1 hour. The
> `get_gcp_access_token()` function above retrieves a *fresh* token at the time
> it is called.
<!-- disableFinding(LINE_OVER_80) -->
> For long-running applications, you implement a refresh mechanism. See [Refresh the access token](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate/openai/auth-and-credentials?hl=en#refresh_your_credentials) for details.
<!-- enableFinding(LINE_OVER_80) -->

### Configuration (Base URL)

<!-- disableFinding(LINE_OVER_80) -->

-   **Global Endpoint** (Recommended for most models requiring global
    availability):
    `https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/global/endpoints/openapi`
-   **Regional Endpoint**:
    `https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/openapi`
    <!-- enableFinding(LINE_OVER_80) -->

### Python Example (OpenMaaS - Chat Completions)

See [`scripts/openmaas_openai_sdk.py`](scripts/openmaas_openai_sdk.py) for the
complete code.

> [!TIP]
> **Alternative: Environment Variables**
> You can set environment variables in your shell instead of updating the code.
>
> ```bash
> export OPENAI_BASE_URL="https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT_ID/locations/global/endpoints/openapi"
> export OPENAI_API_KEY="$(gcloud auth print-access-token)"
> ```
> Then initialize the client without arguments: `client = OpenAI()`

### Python Example (OpenMaaS - Completions API)

The following models support the legacy Completions API: `zai-org/glm-5-maas`,
`moonshotai/kimi-k2-thinking-maas`, `minimaxai/minimax-m2-maas`,
`deepseek-ai/deepseek-v3.1-maas`, and `deepseek-ai/deepseek-v3.2-maas`.

```python
response = client.completions.create(
    model="deepseek-ai/deepseek-v3.2-maas",
    prompt="Once upon a time",
    max_tokens=100
)
print(response.choices[0].text)
```

### Python Example (OpenMaaS - Embeddings)

```python
# Verify specific Embedding Model ID on Model Garden (e.g., intfloat/multilingual-e5-small)
response = client.embeddings.create(
    model="intfloat/multilingual-e5-large-maas",
    input="The quick brown fox jumps over the lazy dog",
)
print(response.data[0].embedding)
```

### Alternative: GenAI SDK

The `google-genai` SDK can also access OpenMaaS models via the `vertexai`
backend.

See [`scripts/openmaas_genai_sdk.py`](scripts/openmaas_genai_sdk.py) for the
complete code.

> [!IMPORTANT]
> **Model ID Format**: For GenAI SDK with OpenMaaS, you **MUST** use the full
> path: `publishers/PUBLISHER/models/MODEL` (e.g.,
> `publishers/zai-org/models/glm-5-maas`).

### Legacy: Agent Platform SDK (OpenMaaS)

For OpenMaaS, you can also use `GenerativeModel` (if supported).

See [`scripts/openmaas_vertexai_sdk.py`](scripts/openmaas_vertexai_sdk.py) for
the complete code.

> [!IMPORTANT]
> **Model ID Format**: For Agent Platform SDK with OpenMaaS, you **MUST** use the
> full path: `publishers/PUBLISHER/models/MODEL`.

### Model Reference & Availability

**Documentation**: [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)

> [!TIP]
> **Self-Deployment for Control**: If you need **dedicated hardware**
> (GPUs/TPUs), **guaranteed capacity**, or **specific regional placement** not
> offered by MaaS, you can **Self-Deploy** these models to Agent Platform
> Endpoints. Search for the model in Model Garden and click "Deploy" to select
> your machine type.

> [!IMPORTANT]
> **Finding Inference Examples**: The list above is a starting point. For the
> **definitive** inference snippets (especially for Chat Completions payload
> structure):
> 1.  Consult the [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
>     list.
> 2.  Click the link for your specific model (e.g., "DeepSeek-V3") to visit its
>     **Model Garden** page.
> 3.  Look for the **"Sample Code"** or **"Use this model"** button on the Model
>     Garden page to get the exact `curl` or Python code for that specific model
>     version.

> [!NOTE]
> This list is **INCOMPLETE**. See [Use Open Models on Agent Platform]
> (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
> for the full list of supported models.

| Model Family | Model ID Examples | Location | Notes |
| :--- | :--- | :--- | :--- |
| **Llama 4** | `meta/llama-4-maverick-17b-128e-instruct-maas` | `us-east5` | |
| **Llama 4** | `meta/llama-4-scout-17b-16e-instruct-maas` | `us-east5` | |
| **Llama 3.3** | `meta/llama-3.3-70b-instruct-maas` | `us-central1` | |
| **DeepSeek** | `deepseek-ai/deepseek-v3.2-maas` | `global` | Global ONLY |
| **DeepSeek** | `deepseek-ai/deepseek-v3.1-maas` | `us-west2` | US-West2 ONLY |
| **DeepSeek** | `deepseek-ai/deepseek-r1-0528-maas` | `us-central1` | |
| **Qwen 3** | `qwen/qwen3-coder-480b-a35b-instruct-maas` | `global` | |
| **Qwen 3** | `qwen/qwen3-next-80b-a3b-instruct-maas` | `global` | |
| **Kimi** | `moonshotai/kimi-k2-thinking-maas` | `global` | |
| **MiniMax** | `minimaxai/minimax-m2-maas` | `global` | |
| **GLM** | `zai-org/glm-4.7-maas`, `zai-org/glm-5-maas` | `global` | |

## 3. Troubleshooting & Common Error Codes

### 429: Resource Exhausted

*   **Cause**: OpenMaaS and Gemini models use **Dynamic Shared Quota (DSQ)**.
    Resources are pooled and allocated dynamically based on availability. A 429
    error indicates the shared pool is temporarily exhausted, not necessarily
    that *your* specific project quota is hit (though it can be).
*   **Solution**: Implement strict **exponential backoff and retry** strategies.
*   **High Throughput**: For production workloads requiring high throughput or guaranteed capacity, consider **Provisioned Throughput (PT)**.
*   **Important**: Quota increases through normal cloud processes (Cloud Console) are **NOT** applicable for DSQ constraints.
*   **Documentation**: [Quotas and limits (DSQ)](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/quotas)

### 400: User Validation Error

*   **Cause**: Invalid request format, unsupported parameter, or incorrect Model ID.
*   **Action**: Double-check your request payload and parameters. Verify the Model ID and Region are correct.

### 404: Not Found / Model Not Available

*   **Cause**: The model is not enabled, or not available in the specified project or region.
*   **Action**:
    1.  **Check Location Availability**:
        *   **OpenMaaS**: Verify the model is available in your region. See [Model Availability by Location](https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#genai-open-models).
        *   **Gemini**:
            <!-- disableFinding(LINE_OVER_80) -->
            *   **Source of Truth**: Always check [Gemini Model Locations](https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#google-models) for the authoritative list.
            <!-- enableFinding(LINE_OVER_80) -->
            *   **Preview Models**: All Preview models (e.g., Gemini 3.1, experimental versions) are often **ONLY** available in the `us-central1` or `global` regions.
            *   **Stable Models**: (e.g., Gemini 2.5 Pro) Available in `us-central1`, `europe-west4`, and many other regions.
            *   **Important**: If you get a 404/400 error, try switching your client location to `us-central1` or `global`.
    2.  **Enable Llama Models**: For **Llama 3.3** and **Llama 4**, you **MUST**
        enable the model in Model Garden before use. Go to the [Model Garden]
        (https://console.cloud.google.com/agent-platform/model-garden), search
        for the model card (e.g., "Llama 3.3 API Service"), and click
        **Enable**. Only then can you make inference requests.

google/skills/tree/main/skills/cloud/agent-platform-inference commit · aabe37a010

▌
Agent Platform Tuning · google bundle
Fine-tune open models or Gemini models using Agent Platform infrastructure, from environment setup through data preparation, job configuration, monitoring, and deployment.

14.4k
repo stars
▌
Agent Platform Prompt Management · google bundle
Creates, lists, retrieves, versions, and deletes managed prompts in Google Cloud Agent Platform using the Python SDK.

14.4k
repo stars
▌
Agent Platform Alert Configuration · google bundle
Configures dynamic threshold alerting policies for Google Cloud Vertex AI Agent Platform agents, monitoring latency, error rates, and quality metrics using Terraform and PromQL.

14.4k
repo stars
▌
Agent Platform Endpoint Management · google
Manages Agent Platform serving endpoints: create, list, describe, update, delete, and troubleshoot permission, quota, and resource busy errors.

14.4k
repo stars
▌
Agent Platform Model Registry · google
Manage machine learning models in the Agent Platform Model Registry: list, describe, upload, update, and delete models and their versions.

14.4k
repo stars
▌
Gemini API · google bundle
Guides usage of the Gemini API on Agent Platform with the Google Gen AI SDK, covering SDK usage (Python, JS/TS, Go, Java, C#), capabilities like multimodal inputs, tools, media generation, caching, batch prediction, and Live API.

14.4k
repo stars

Frequently asked questions

How do I install the Agent Platform Inference skill?

Run npx skillmds add google/agent-platform-inference in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Agent Platform Inference skill do?

Is Agent Platform Inference safe to use?

SkillMD's automated safety review verdict for this skill is CAUTION. Independent scanners report: SkillSpector: WARNING, Skill Scanner: PASS. Capability flags: executes scripts, makes network calls, reads secrets. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Agent Platform Inference?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Agent Platform Inference free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Agent Platform Inference?

Google (@google) published this skill as a verified publisher. Their other Agent Skills are listed on their SkillMD profile.

Agent Platform Inference

Agent Platform GenAI Inference Skill

Safety & Confirmation Tiers (CRITICAL)

Phase 0: Environment Setup

Workflow Decision Tree

1. Gemini Models

Choosing the Right SDK

Installation

Python Example (GenAI SDK - Preferred)

Alternative: OpenAI SDK (Chat Completions)

Legacy: Agent Platform SDK

2. OpenMaaS Models (Llama, DeepSeek, Qwen, etc.)

Installation

Authentication for OpenAI SDK

Configuration (Base URL)

Python Example (OpenMaaS - Chat Completions)

Python Example (OpenMaaS - Completions API)

Python Example (OpenMaaS - Embeddings)

Alternative: GenAI SDK

Legacy: Agent Platform SDK (OpenMaaS)

Model Reference & Availability

3. Troubleshooting & Common Error Codes

429: Resource Exhausted

400: User Validation Error

404: Not Found / Model Not Available

Related

Frequently asked questions