# 2483 Llms Txt E9bdd61a

> Vastai-Api - Llms-Txt

- Skill: `tools-only/2483-llms-txt-e9bdd61a` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add tools-only/2483-llms-txt-e9bdd61a`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/2483-llms-txt-e9bdd61a/raw
- Safety review: pending (external: skill-scanner PASS, skillspector FAIL)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: tools-only (https://skillmd.com/u/tools-only)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/tools-only/2483-llms-txt-e9bdd61a

---

# Vastai-Api - Llms-Txt

**Pages:** 283

---

## Workergroup Parameters

**URL:** llms-txt#workergroup-parameters

**Contents:**
- gpu\_ram
- launch\_args
- search\_params
- template\_hash
- template\_id
- test\_workers

The following parameters can be specified specifically for a Workergroup and override Endpoint parameters. The Endpoint parameters will continue to apply for other Workergroups contained in it, unless specifically set.&#x20;

* min\_load
* target\_util
* cold\_mult

The parameters below are specific to only Workergroups, not Endpoints.

The amount of GPU memory (VRAM) in gigabytes that your model or workload requires to run. This parameter tells the serverless engine how much GPU memory your model needs.

If not specified during workergroup creation, the default value is 24.

A command-line style string containing additional parameters for instance creation that will be parsed and applied when the serverless engine creates new workers. This allows you to customize instance configuration beyond what's specified in templates.

There is no default value for launch\_args.

A query string, list, or dictionary that specifies the hardware and performance criteria for filtering GPU offers in the vast.ai marketplace. It uses a simple query syntax to define requirements for the machines that your Workergroup will consider when searching for workers to create.

There is no default value for search\_params. To see all available search filters, see the CLI docs [here](https://docs.vast.ai/cli/commands).

A unique hexadecimal identifier that references a pre-configured template containing all the configuration needed to create instances. Templates are comprehensive specifications that include the Docker image, environment variables, onstart scripts, resource requirements, and other deployment settings.

There is no default value for template\_hash.

A numeric (integer) identifier that uniquely references a template in the Vast.ai database. This is an alternative way to reference the same template that `template_hash` points to, but using the template's database primary key instead of its hash string.

There is no default value for template\_id.

The number of different physical machines that a Workergroup should test during its initial "exploration" phase to gather performance data before transitioning to normal demand-based scaling. The Worker Group remains in "exploring" mode until it has successfully tested at least `floor(test_workers / 2)` machines.

If not specified during workergroup creation, the default value is 3.

---

## delete endpoint

**URL:** llms-txt#delete-endpoint

Source: https://docs.vast.ai/api-reference/serverless/delete-endpoint

api-reference/openapi.json delete /api/v0/endptjobs/{id}/
Deletes an endpoint group by ID. Associated workergroups will also be deleted.

CLI Usage: `vastai delete endpoint <id>`

---

## QuickStart

**URL:** llms-txt#quickstart

**Contents:**
  - 1. Sign Up & Add Credit
  - &#x32;**. Prepare to Connect**
  - &#x33;**. Pick a&#x20;**[**Template**](/documentation/instances/templates)**&#x20;& Find a Machine**
  - **4.&#x20;**[**Manage or End Your Instance**](/documentation/instances/managing-instances)
- Common Questions
  - What is a minimum deposit amount?
  - What happens when my balance runs out? Can I avoid interruptions?
  - How can I customize a template?

Source: https://docs.vast.ai/documentation/get-started/quickstart

<script
  type="application/ld+json"
  dangerouslySetInnerHTML={{
__html: JSON.stringify({
  "@context": "https://schema.org",
  "@type": "HowTo",
  "name": "How to Get Started with Vast.ai",
  "description": "A step-by-step guide to setting up your Vast.ai account and running your first GPU instance.",
  "step": [
    {
      "@type": "HowToStep",
      "name": "Sign Up & Add Credit",
      "text": "Create an account on vast.ai, verify your email address, and add credit through Billing using credit card, Coinbase, or Crypto.com. Your balance will appear at the top right of the dashboard."
    },
    {
      "@type": "HowToStep",
      "name": "Prepare to Connect",
      "text": "For SSH access: generate an SSH key pair and upload your public key in the Keys page. For Jupyter access: download and install the provided TSL certificate for secure browser access."
    },
    {
      "@type": "HowToStep",
      "name": "Pick a Template & Find a Machine",
      "text": "Browse Templates for pre-built setups like PyTorch, TensorFlow, or ComfyUI. Go to Search and filter by GPU type, count, RAM, CPU, network speed, and price. Remember that disk space is permanent and cannot be changed later. Click Rent when you find a match and wait for the instance to start."
    },
    {
      "@type": "HowToStep",
      "name": "Manage or End Your Instance",
      "text": "Use Stop to pause GPU billing (storage still accrues charges). Use Delete when finished to stop all charges."
    }
  ]
})
}}
/>

<script
  type="application/ld+json"
  dangerouslySetInnerHTML={{
__html: JSON.stringify({
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What is a minimum deposit amount?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "The minimum deposit amount on Vast.ai is $5."
      }
    },
    {
      "@type": "Question",
      "name": "What happens when my balance runs out? Can I avoid interruptions?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "When your balance reaches zero, your running instances will automatically stop. To avoid this, you can enable auto-billing on the Billing page. Set an auto-charge threshold higher than your average daily spend, so your card is automatically charged when your balance falls below that amount. We also recommend setting a low-balance email alert at a slightly lower threshold to notify you if the auto-charge fails for any reason."
      }
    },
    {
      "@type": "Question",
      "name": "How can I customize a template?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "You can create a new template from scratch, or you can edit an existing template. You can find a guide in the templates documentation."
      }
    }
  ]
})
}}
/>

This Quickstart will guide you through setting up your Vast.ai account and running your first instance in just a few steps.&#x20;

### 1. Sign Up & Add Credit

* Create an account on [vast.ai.](https://cloud.vast.ai/)
* Verify your email address.
* Go to [**Billing**](https://cloud.vast.ai/billing/) → **Add Credit** and top up using credit card, Coinbase, or Crypto.com.

<img src="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=3512036ce752f432218c5fc67adfe073" alt="Billing" data-og-width="800" width="800" data-og-height="853" height="853" data-path="images/guides-overview-quick-start.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=280&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=232b461125a3d4a98ab962094e77a166 280w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=560&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=9cd76de9c4d957a0fb632a9f87465ef1 560w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=840&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=683729ddfc9debc85e0809fcff5d737e 840w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=1100&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=3555ef813dc1ae188112c85f2b058bcb 1100w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=1650&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=f5a88f091c569857fda64388c837fb61 1650w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start.webp?w=2500&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=6d9a579281e68580e6a6482eda020f87 2500w" />
* Your balance appears at the top right of the dashboard.

<img src="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=024d3a562b305ac29f47cea16022536f" alt="" data-og-width="973" width="973" data-og-height="65" height="65" data-path="images/guides-overview-quick-start-2.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=280&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=3deed130eecbc43adb385e88452fa94f 280w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=560&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=25fe2a5f2883f1aa4ea55f2df7f76ec1 560w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=840&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=6894eea5311426107efcd81d683c8531 840w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=1100&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=532c990e02204e56dc8a9767ef03d5a2 1100w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=1650&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=a8dae300bcde252e34ab7d47c3de3362 1650w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-2.webp?w=2500&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=a2d8dd66a18b678322a2f5b34002a3dd 2500w" />

<Note>
  Before you can **rent a machine&#x20;**&#x6F;r **create a team**, you must verify your email address. After signing up, check your inbox (and spam folder) for the verification email  and click the link inside. You can resend the verification email anytime from **Settings → Resend Verification Email.**
</Note>

### &#x32;**. Prepare to Connect**

* **For SSH access**: generate an [SSH key pair](/documentation/instances/sshscp) and upload your **public key** in [Keys page](https://cloud.vast.ai/manage-keys/).

<img src="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=5b7bd06c9d287c89754a598a2be3c360" alt="Keys" data-og-width="1279" width="1279" data-og-height="579" height="579" data-path="images/guides-overview-quick-start-3.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=280&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=b2b2dd3126a1a716ba6d6e70832d9e99 280w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=560&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=fa2b0ef4be9a051694ee4e28f9263ce1 560w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=840&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=6faf68e85e85941c6afcd85602af3613 840w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=1100&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=3864b38246ed01a168df7ef5357a9349 1100w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=1650&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=6b492e5d08306470905e63dce37a3dc2 1650w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-3.webp?w=2500&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=4200de798dd314922cec733c92b4160b 2500w" />
* **For Jupyter access**: download and install the provided [TSL certificate](/documentation/instances/jupyter#1SmCz) (needed for secure browser access).

<Note>
  If you don’t install the provided browser certificate:&#x20;

* **Windows / Linux** – You’ll see a **“Your connection is not private”** privacy warning. You can still connect by clicking **Advanced** → **Proceed**, but the warning will appear every time.
  * **macOS** – Browsers will block Jupyter until you install and trust the provided certificate in **Keychain Access**. Without it, you won’t be able to connect.

Installing the certificate once removes the warning permanently.
</Note>

### &#x33;**. Pick a&#x20;**[**Template**](/documentation/instances/templates)**&#x20;& Find a Machine**

* Browse [**Templates**](https://cloud.vast.ai/templates/) for pre-built setups (e.g., [PyTorch](/pytorch), TensorFlow, ComfyUI).
* Go to [**Search**](https://cloud.vast.ai/create/) and filter by GPU type, count, RAM, CPU, network speed, and price.
* **Disk Space is Permanent.&#x20;**&#x54;he disk size you choose when creating an instance cannot be changed later. If you run out of space, you’ll need to create a new instance with a larger disk. Tip: Allocate a bit more than you think you need to avoid interruptions.
* Click **Rent** when you find a match.
* Wait for the instance to start—cached images launch quickly, fresh pulls may take 10–60 minutes.
* Click **Open** button to access your instance.

### **4.&#x20;**[**Manage or End Your Instance**](/documentation/instances/managing-instances)

* Use **Stop** to pause GPU billing (storage still accrues charges).
* Use **Delete** when finished to stop *all* charges.

<img src="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=5924a1a46183b876b909daf802c07841" alt="Manage or End Your Instance" data-og-width="1018" width="1018" data-og-height="249" height="249" data-path="images/guides-overview-quick-start-4.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=280&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=f08910728e4eebb9dd822c866cb6a602 280w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=560&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=b48d2026679f1e83ccaab2ce2adc4e2f 560w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=840&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=6f48b1a4ee377ee27d8a67ec83f770f1 840w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=1100&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=4c7f2ecefa1c7870b45336ed4ec86820 1100w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=1650&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=8766a3a91186dd38eef4d2e2d97af3e1 1650w, https://mintcdn.com/vastai-80aa3a82/KWV1TCiq0zjC57hU/images/guides-overview-quick-start-4.webp?w=2500&fit=max&auto=format&n=KWV1TCiq0zjC57hU&q=85&s=ebe0bf0b9e5da78c1e65a3c50273f97c 2500w" />

### What is a minimum deposit amount?

The minimum deposit amount on Vast.ai is \$5.

### What happens when my balance runs out? Can I avoid interruptions?

When your balance reaches zero, your running instances will automatically stop. To avoid this, you can enable **auto-billing&#x20;**&#x6F;n the Billing page. Set an auto-charge threshold higher than your average daily spend, so your card is automatically charged when your balance falls below that amount. We also recommend setting a **low-balance email alert&#x20;**&#x61;t a slightly lower threshold to notify you if the auto-charge fails for any reason.&#x20;

### How can I customize a template?

You can create a new template from scratch, or you can edit an existing template. You can find a guide [here](/documentation/instances/templates#LrOME).&#x20;

---

## Huggingface TGI with LLama3

**URL:** llms-txt#huggingface-tgi-with-llama3

**Contents:**
- 1) Choose The Huggingface LLama3 TGI API Template From the Recommended Section
- 2) Modifying the Template
- 3) Rent a GPU
- 4) Monitor Your Instance
- 5) Congratulations!

Source: https://docs.vast.ai/huggingface-tgi-with-llama3

This is a guide on how to setup and expose an API for Llama3 Text Generation.

## 1) Choose The Huggingface LLama3 TGI API Template From the Recommended Section

Login to your Vast account on the [console](https://cloud.vast.ai)

Select the [HuggingFace Llama3 TGI API](https://cloud.vast.ai/?template_id=906891f677fb36f21662a92e6092b5fc) template by clicking the link provider

For this template we will be using the meta-llama/Meta-Llama-3-8B-Instruct model, and the TGI 2.0.4 from Huggingface

Templates encapsulate all the information required to run an application with the autoscaler, including machine parameters, docker image, and environment variables.

For this template, the only requirement is that you have your own Huggingface access token. You will also need to apply to have access to Llama3 on huggingface in order to access this gated repository.

The template comes with some filters that are minimum requirements for TGI to run effectively. This includes but is not limited to a disk space requirement of 100GB, and a gpu ram requirement of at least 16GB.

After selecting the template your screen should look like this:

<Frame caption="Select">
    <img src="https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=02e8297d55467706c3411dcb8a9fa6bf" alt="Select" data-og-width="1082" width="1082" data-og-height="465" height="465" data-path="images/Select.png" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=280&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=8fff2658ba8f39ea17b1244a3e74e88f 280w, https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=560&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=8fb7d9f1ddde6e1be705793093308c26 560w, https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=840&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=60bc97e88501c1fd1a8ee3de9a455517 840w, https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=1100&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=3baf5713662c147ee48d9e7cf85d16fe 1100w, https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=1650&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=7a6e17fad2f5bab89d29bd4fb4cd9248 1650w, https://mintcdn.com/vastai-80aa3a82/MV3o3hbz7ZsjLzLy/images/Select.png?w=2500&fit=max&auto=format&n=MV3o3hbz7ZsjLzLy&q=85&s=4928a1b1b5bc5569ac5575d94cbd7085 2500w" />
</Frame>

## 2) Modifying the Template

Once you have selected the template, you will need to then add in your huggingface token and click the 'Select & Save' button.

You can add your huggingface token with the rest of the docker run options.

<Frame caption="Edithf">
  ![Edithf](https://vast.ai/uploads/HuggingFace/EditHf.png)
</Frame>

This is the only modification you will need to make on this template.

You can then press 'Select & Save' to get ready to launch your instance.

Once you have selected the template, you can then choose to rent a GPU of your choice from either the search page or the CLI/API.

For someone just getting started I recommend either an Nvidia RTX 4090, or an A5000.

<Frame caption="Rent">
  ![Rent](https://vast.ai/uploads/HuggingFace/Rent.png)
</Frame>

## 4) Monitor Your Instance

Once you rent a GPU your instance will being spinning up on the Instances page.

You know the API will be ready when your instance looks like this:

<Frame caption="Llama3Tgiinstances">
  ![Llama3Tgiinstances](https://vast.ai/uploads/llama3tgiinstances.png)
</Frame>

Once your instance is ready you will need to find where your API is exposed. Go to the IP & Config by pressing the blue button on the top of the instance card. You can see the networking configuration here.

<Frame caption="Llama3Ip">
  ![Llama3Ip](https://vast.ai/uploads/llama3ip.png)
</Frame>

After opening the IP & Port Config you should see a forwarded port from 5001, this is where your API resides. To hit TGI you can use the '/generate' endpoint on that port.

<Frame caption="Llama3Tgipostman">
  ![Llama3Tgipostman](https://vast.ai/uploads/llama3tgipostman.png)
</Frame>

## 5) Congratulations!

You now have a running instance with an API that is using TGI loaded up with Llama3 8B!

---

## SSH Connection

**URL:** llms-txt#ssh-connection

**Contents:**
- About SSH
- Quick start: Generate and add your SSH key to your Vast account
- Connecting to your Instance
  - Direct vs Proxy Connections
- Tmux
- SSH Local Port Forwarding
- SSH Alternative - Jupyter Terminal
- Troubleshooting
  - Permission Denied (publickey)
  - SSH Key Changes

Source: https://docs.vast.ai/documentation/instances/connect/ssh

Learn how to securely connect to Vast.ai instances using SSH. Generate keys, establish connections, use port forwarding, and integrate with VS Code.

**SSH (Secure Shell)** is a protocol for safely connecting to remote servers. It encrypts your connection so you can:

* Log in securely
* Run commands remotely
* Transfer files without exposing your data

<Note>
  Vast.ai instances are configured to accept keys only - Password authentication is disabled for improved security.
</Note>

## Quick start: Generate and add your SSH key to your Vast account

<Tabs>
  <Tab title="Terminal">
    **1. Generate a SSH key pair in your terminal**

1. Creates two files (by default in \~/.ssh/):
       * id\_ed25519 → your **private key** (keep safe, never share).
       * id\_ed25519.pub → your **public key** (safe to share, add to servers).
    2. -C "[your\_email@example.com](mailto:your_email@example.com)" is optional. Whatever you put there is stored as a comment in the public key file (e.g., id\_ed25519.pub). It's just for identification (helpful if you use multiple keys), not for security.

<Note>
      When you run ssh-keygen -t ed25519 in **Windows PowerShell**, the keys are created in your Windows user profile folder:
      `C:\Users\<YourUsername>\.ssh\`
    </Note>

**2. Copy your public key.**

**3. Add it in your** [**vast account**](https://cloud.vast.ai/manage-keys/)

<img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=0c4875e49e2b1250de56ca5d06c8dd8a" alt="" data-og-width="914" width="914" data-og-height="684" height="684" data-path="images/instances-sshscp.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=ce25808c68ef5747ae61afc47e21dbac 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=ad998abfde5ababac4ce81f09fd2d465 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=d02a30b53bd09b1014350478397dece5 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=8154e85ec34d201cde1ae4647f334bec 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=6577c4d72389e767fb8ffe14e92167f8 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=7e4c0d56416402102e8e115bc9c481e6 2500w" />
  </Tab>

<Tab title="Vast CLI">
    **Add & Generate SSH Key (using** [**Vast CLI**](/cli/get-started)**)**

1. **Install Vast CLI:**

2. **Generate an API key in your vast account:**
       1. Open [CLI page](https://cloud.vast.ai/cli/)
       2. Create an API key
          <Frame caption="API Key creation">
              <img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=b87717158bb150f91e707186ee5e3d0f" alt="API Key creation" data-og-width="800" width="800" data-og-height="316" height="316" data-path="images/instances-sshscp-2.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=142b3c9d5f6649159213ffd6808b72dd 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=00b017024671fc23e9ff633b929ad068 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=29ff3c6567e5eb5fc92198a007d2cbb8 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=cff4606483c2ce7b077be309ed80af02 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=b154d75102218364a8a476b999b003dd 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-2.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=5959f2128d22d7794176c4fa391d30dd 2500w" />
          </Frame>

3. **Generate a new SSH key pair** (you will need your vast API key):

* Saves keys as \~/.ssh/id\_ed25519 (private) and \~/.ssh/id\_ed25519.pub (public).
    * Backs up existing keys as .backup\_\[timestamp].
    * Keys are stored in your Vast account and used for new instances.
  </Tab>
</Tabs>

<Warning>
  * Adding a key to your account keys only applies to **new instances**.
  * Existing instances will **not** get the new key automatically. To add a key, use the **instance-specific SSH interface**.
  * For **VM instances**, changing keys requires recreating the VM.
</Warning>

## Connecting to your Instance

Start a new instance and click the SSH icon to see your connection information.

<Frame caption="Terminal Connection Options">
    <img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=0baecee39ed1714254241d265a1583ef" alt="Connection details" data-og-width="1063" width="1063" data-og-height="453" height="453" data-path="images/instances-sshscp-3.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=23a7cc28f35a45da60ae7bac6851c5c3 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=55d399312c5cd759898b27d23d80fd05 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=46cf12ad1ed7a5fa4dc4e0f377c64059 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=8a954169f77ccb3e9ff2ce5d095d00e8 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=8d694128da14e5c0540821be56579377 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-3.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=7cfb49180130e1d6dd523d2d90c9fe99 2500w" />
</Frame>

Now you can enter the connection command string into your terminal

You should now see a screen similar to this. You will, by default, be placed into a tmux session.

<Frame caption="Connected to Instance">
    <img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=e3dadd004d54ec00fbe7236860f5adc2" alt="Instance SSH session" data-og-width="1254" width="1254" data-og-height="464" height="464" data-path="images/instances-sshscp-4.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=a185fcc096f446ee40f48d6bc58a8871 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=fabdfe7e040ea540d21dc4c552966358 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=3d35d7fb1b7d2d5a1bd7731a02d5dff2 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=e259ec3f325fa860ba277948a5481ce8 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=53eb278140b2d8dd6507121fcaada6ee 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-4.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=29713333a4bf70aac4f71d5d96fb8bee 2500w" />
</Frame>

### Direct vs Proxy Connections

Vast offers both proxy (default) and direct connection methods for SSH:

* **Proxy SSH**: Works on all machines, slower for data transfer, uses Vast proxy server
* **Direct SSH**: Requires machines with open ports, faster and more reliable, preferred method

We connect you to a tmux session by default for reliability and to prevent unintentional termination of foreground processes. You can create a new bash terminal window with `ctrl+b` + `c`. Cycle through your windows with `ctrl+b` + `n`

There is an excellent guide for getting to grips with tmux at [https://tmuxcheatsheet.com](https://tmuxcheatsheet.com/)

If, however, you would prefer to disable TMUX, you can apply the following either in a terminal or from your template's on-start section.

## SSH Local Port Forwarding

An often overlooked feature of SSH is its ability to forward local ports to another machine. When you access a server remotely over SSH, you can make ports from the remote machine available as if they were listening on your own device. This is a secure alternative to opening ports on the public interface as all data is transported over the SSH connection.

This SSH command connects to the remote instance and sets up **local port forwarding** (SSH tunneling):

**Connection details:**

* Connects to IP 180.123.123.123 as user root
* Uses port 1234 instead of the default SSH port 22

**Port forwarding (the key part):**

* `-L 8080:localhost:8080` - Creates a tunnel so when you access localhost:8080 on your local machine, it forwards to port `8080` on the remote server
* `-L 5000:localhost:5000` - Same thing for port `5000`

You can repeat the `-L` arguments to forward as many ports as you need.

**What this means:** After connecting, you can open your web browser and go to [https://localhost:8080](https://localhost:8080) or [http://localhost:5000](http://localhost:5000) on your local computer, and you'll actually be accessing services running on those ports on the remote server. It's like creating secure "tunnels" through the SSH connection to reach applications on the remote machine that might not be directly accessible from the internet.

## SSH Alternative - Jupyter Terminal

As a simple alternative to SSH, you might like to consider Jupyter Terminal instead. All instances started in Jupyter launch mode will have this enabled. It is a very straightforward web-based terminal with session persistence. It's great for a quick CLI session.

Access the terminal from the SSH connections interface.

<img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=209bceca0c9c25960269d6af03a4ec00" alt="" data-og-width="800" width="800" data-og-height="174" height="174" data-path="images/instances-sshscp-5.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=7fd1f0b63e9e32976482c161d8adb0ff 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=3a37e3391a09f2b2d5442640ad5371a1 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=445a424b204acae3aa5892c0fb5e19f7 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=3a1c02fd93343b38f04db5301a07abe1 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=9f74b697ddbcd0b226db17785c72ff8d 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp-5.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=19409536de358a89230eeb9f50f1683c 2500w" />

### Permission Denied (publickey)

If you get this error when trying to SSH:

1. Ensure your SSH key is added to your [Vast account](https://cloud.vast.ai/manage-keys/)
2. Verify you're using the correct private key
3. Check key file permissions: `chmod 600 ~/.ssh/id_ed25519`
4. Use `-vv` flag for detailed debug info: `ssh -vv -p PORT root@IP`

* New account keys only apply to NEW instances created after adding the key
* Existing instances keep their original keys (won't get new keys automatically)
* For VM instances, changing keys requires recreating the VM
* To add keys to existing instances, use the instance-specific SSH interface

### General Connection Issues

You can often determine the exact cause of a connection failure by using the -vv arguments with ssh to get more information.

Common reasons include:

* Using the wrong private key
* Incorrect permissions for your private key
* Public key not added to instance or account
* Connecting to the wrong port

## SCP & SFTP File Transfer

Both **SCP** (Secure Copy Protocol) and **SFTP** (SSH File Transfer Protocol) are tools for securely transferring files that piggyback on the SSH protocol. They use the same authentication and encryption as SSH.

### SCP (Secure Copy Protocol)

* **What it is:** Simple, command-line tool for copying files between local and remote machines
* **Best for:** Quick, one-time file transfers
* **Syntax:** `scp -P <port> source destination`

```bash Bash theme={null}

**Examples:**

Example 1 (unknown):
```unknown

```

Example 2 (unknown):
```unknown
</CodeGroup>

    1. Creates two files (by default in \~/.ssh/):
       * id\_ed25519 → your **private key** (keep safe, never share).
       * id\_ed25519.pub → your **public key** (safe to share, add to servers).
    2. -C "[your\_email@example.com](mailto:your_email@example.com)" is optional. Whatever you put there is stored as a comment in the public key file (e.g., id\_ed25519.pub). It's just for identification (helpful if you use multiple keys), not for security.

    <Note>
      When you run ssh-keygen -t ed25519 in **Windows PowerShell**, the keys are created in your Windows user profile folder:
      `C:\Users\<YourUsername>\.ssh\`
    </Note>

    **2. Copy your public key.**

    <CodeGroup>
```

Example 3 (unknown):
```unknown

```

Example 4 (unknown):
```unknown
</CodeGroup>

    **3. Add it in your** [**vast account**](https://cloud.vast.ai/manage-keys/)

        <img src="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=0c4875e49e2b1250de56ca5d06c8dd8a" alt="" data-og-width="914" width="914" data-og-height="684" height="684" data-path="images/instances-sshscp.webp" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=280&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=ce25808c68ef5747ae61afc47e21dbac 280w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=560&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=ad998abfde5ababac4ce81f09fd2d465 560w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=840&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=d02a30b53bd09b1014350478397dece5 840w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=1100&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=8154e85ec34d201cde1ae4647f334bec 1100w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=1650&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=6577c4d72389e767fb8ffe14e92167f8 1650w, https://mintcdn.com/vastai-80aa3a82/rJSuh7NkZ00k8fzX/images/instances-sshscp.webp?w=2500&fit=max&auto=format&n=rJSuh7NkZ00k8fzX&q=85&s=7e4c0d56416402102e8e115bc9c481e6 2500w" />
  </Tab>

  <Tab title="Vast CLI">
    **Add & Generate SSH Key (using** [**Vast CLI**](/cli/get-started)**)**

    1. **Install Vast CLI:**

       <CodeGroup>
```

---

## unlist volume

**URL:** llms-txt#unlist-volume

Source: https://docs.vast.ai/api-reference/volumes/unlist-volume

api-reference/openapi.json post /api/v0/volumes/unlist/
Remove a volume listing from the marketplace.

CLI Usage: `vastai unlist volume <volume_id>`

---

## streaming response from model API to client

**URL:** llms-txt#streaming-response-from-model-api-to-client

class GenerateStreamHandler(EndpointHandler[InputData]):
    @property
    def endpoint(self) -> str:
        return "/generate_stream"

@classmethod
    def payload_cls(cls) -> Type[InputData]:
        return InputData

def generate_payload_json(self, payload: InputData) -> Dict[str, Any]:
        return dataclasses.asdict(payload)

def make_benchmark_payload(self) -> InputData:
        return InputData.for_test()

async def generate_client_response(
        self, client_request: web.Request, model_response: ClientResponse
    ) -> Union[web.Response, web.StreamResponse]:
        match model_response.status:
            case 200:
                log.debug("Streaming response...")
                res = web.StreamResponse()
                res.content_type = "text/event-stream"
                await res.prepare(client_request)
                async for chunk in model_response.content:
                    await res.write(chunk)
                await res.write_eof()
                log.debug("Done streaming response")
                return res
            case code:
                log.debug("SENDING RESPONSE: ERROR: unknown code")
                return web.Response(status=code)

---

## remove defjob

**URL:** llms-txt#remove-defjob

Source: https://docs.vast.ai/api-reference/machines/remove-defjob

api-reference/openapi.json delete /api/v0/machines/{machine_id}/defjob/
Deletes the default job (background instances) for a specified machine.

CLI Usage: `vastai remove defjob <machine_id>`

---

## Creating Templates for GROBID

**URL:** llms-txt#creating-templates-for-grobid

**Contents:**
- Introduction
- Find The Image and Tag You Want to Use
  - Step 1 - Find a Suitable Image
  - Step 2 - Selecting the Version Tag
- Configuring The Template
  - Step 1 - Setting Your Chosen Image and Tag in Your Vast.ai Template
  - Step 2 - Map Ports and Specify Your Image and Tag Combination
  - Step 3 - Select the Launch Mode
  - Step 4 - Look for CMD or ENTRYPOINT command
  - Step 5 - Fill Out On-start Script section using the CMD command we just found

Source: https://docs.vast.ai/documentation/templates/examples/grobid

This guide demonstrates creating a template using an existing Docker image. See our [Creating Templates](/documentation/templates/creating-templates) guide for more details on template configuration. We will be using the image from [GROBID on dockerhub](https://hub.docker.com/r/grobid/grobid).

## Find The Image and Tag You Want to Use

### Step 1 - Find a Suitable Image

There are multiple GROBID images in dockerhub, but for this guide we will be using the official GROBID image.

<Frame caption="Grobid Overview">
  ![Grobid

…(truncated)
