Vastai-Api - Llms-Txt

Vastai-Api - Llms-Txt

Pages: 283

Workergroup Parameters

URL: llms-txt#workergroup-parameters

Contents:

• gpu_ram
• launch_args
• search_params
• template_hash
• template_id
• test_workers

The following parameters can be specified specifically for a Workergroup and override Endpoint parameters. The Endpoint parameters will continue to apply for other Workergroups contained in it, unless specifically set.

• min_load
• target_util
• cold_mult

The parameters below are specific to only Workergroups, not Endpoints.

The amount of GPU memory (VRAM) in gigabytes that your model or workload requires to run. This parameter tells the serverless engine how much GPU memory your model needs.

If not specified during workergroup creation, the default value is 24.

A command-line style string containing additional parameters for instance creation that will be parsed and applied when the serverless engine creates new workers. This allows you to customize instance configuration beyond what's specified in templates.

There is no default value for launch_args.

A query string, list, or dictionary that specifies the hardware and performance criteria for filtering GPU offers in the vast.ai marketplace. It uses a simple query syntax to define requirements for the machines that your Workergroup will consider when searching for workers to create.

There is no default value for search_params. To see all available search filters, see the CLI docs here.

A unique hexadecimal identifier that references a pre-configured template containing all the configuration needed to create instances. Templates are comprehensive specifications that include the Docker image, environment variables, onstart scripts, resource requirements, and other deployment settings.

There is no default value for template_hash.

A numeric (integer) identifier that uniquely references a template in the Vast.ai database. This is an alternative way to reference the same template that template_hash points to, but using the template's database primary key instead of its hash string.

There is no default value for template_id.

The number of different physical machines that a Workergroup should test during its initial "exploration" phase to gather performance data before transitioning to normal demand-based scaling. The Worker Group remains in "exploring" mode until it has successfully tested at least floor(test_workers / 2) machines.

If not specified during workergroup creation, the default value is 3.

delete endpoint

URL: llms-txt#delete-endpoint

Source: https://docs.vast.ai/api-reference/serverless/delete-endpoint

api-reference/openapi.json delete /api/v0/endptjobs/{id}/ Deletes an endpoint group by ID. Associated workergroups will also be deleted.

CLI Usage: vastai delete endpoint <id>

QuickStart

URL: llms-txt#quickstart

Contents:

• 1. Sign Up & Add Credit
• 2. Prepare to Connect
• 3. Pick a Template & Find a Machine
• 4. Manage or End Your Instance
• Common Questions
  • What is a minimum deposit amount?
  • What happens when my balance runs out? Can I avoid interruptions?
  • How can I customize a template?

Source: https://docs.vast.ai/documentation/get-started/quickstart