Deploy Infrastructure

Deploy HyperPod EKS Infrastructure

Overview

Use this skill to deploy the HyperPod EKS infrastructure that hosts the Slurm cluster. This is Phase 1 of the slinky-slurm deployment workflow and must complete before running setup.sh or install.sh.

The deploy.sh script supports two infrastructure backends:

• CloudFormation (--infra cfn) -- uses an AWS-hosted S3 template
• Terraform (--infra tf) -- uses local Terraform modules

Deployment takes approximately 20-30 minutes and provisions:

• VPC with public/private subnets
• EKS cluster
• HyperPod instance groups (management + accelerated)
• FSx Lustre file system
• Security groups and IAM roles

Prerequisites

Before running deploy.sh, verify:

1. AWS CLI installed and configured with valid credentials
2. jq installed (for --infra cfn) or terraform installed (for --infra tf)
3. A valid AWS region and AZ ID for instance placement
4. No existing stack with the same name (default: hp-eks-slinky-stack)

See the deployment-preflight skill for detailed prerequisite validation.

Steps

Step 1: Choose instance type and infrastructure backend

Any SageMaker-supported GPU instance type can be used. GPU count, model, and EFA interfaces are auto-discovered via the EC2 API. Two common configurations:

Step 2: Run deploy.sh

CloudFormation path (recommended for first-time users):

# Deploy g5 in us-west-2 (defaults)
bash deploy.sh --instance-type ml.g5.8xlarge --infra cfn

# Deploy p5 in us-east-1 with a specific AZ
bash deploy.sh --instance-type ml.p5.48xlarge --instance-count 2 --infra cfn --region us-east-1 --az-id use1-az2

# Deploy with custom stack name
bash deploy.sh --instance-type ml.g5.8xlarge --infra cfn --stack-name my-slinky-stack

Terraform path:

# Deploy g5 using Terraform
bash deploy.sh --instance-type ml.g5.8xlarge --infra tf

# Deploy p5 in us-east-1
bash deploy.sh --instance-type ml.p5.48xlarge --instance-count 2 --infra tf --region us-east-1 --az-id use1-az2

The Terraform path will show a plan and prompt for confirmation before applying.

Step 3: Source environment variables

After deploy.sh completes, it writes env_vars.sh containing stack outputs:

source env_vars.sh

This sets:

• AWS_ACCOUNT_ID
• AWS_REGION
• EKS_CLUSTER_NAME
• VPC_ID
• PRIVATE_SUBNET_ID
• SECURITY_GROUP_ID
• STACK_ID

Step 4: Update kubeconfig

aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $AWS_REGION

Step 5: Verify cluster connectivity

kubectl cluster-info
kubectl get nodes

Expected output: nodes with instance types matching the chosen profile (ml.m5.4xlarge for management, ml.g5.8xlarge or ml.p5.48xlarge for accelerated).

What deploy.sh Does Internally

CloudFormation Path

1. Sources lib/deploy_helpers.sh
2. Calls resolve_instance_profile() to set instance type/count for the profile
3. Validates AWS credentials via aws sts get-caller-identity
4. Resolves AZ IDs for the region via aws ec2 describe-availability-zones
5. Validates the specified --az-id against the resolved list
6. Reads params.json (40 CloudFormation parameters, g5 defaults)
7. Calls resolve_cfn_params() to substitute AZ IDs and instance overrides
8. Calls validate_cfn_template() to:
   • Validate the S3-hosted template is reachable and syntactically valid via aws cloudformation validate-template --template-url
   • Cross-check parameter keys in the resolved params against the template's declared parameters
   • Error if required parameters (no default value) are missing
   • Warn if extra parameters are provided that the template doesn't expect
9. Calls aws cloudformation create-stack (or update-stack for existing stacks) with the S3-hosted HyperPod template
10. Waits for stack completion (20-30 minutes)
11. Extracts outputs to env_vars.sh

Terraform Path

1. Same initial steps (1-5) as CloudFormation
2. Copies custom.tfvars to the terraform-modules directory
3. Calls resolve_tf_vars() to patch region, AZ, and instance overrides
4. Runs terraform init, plan, and apply (with user confirmation)
5. Runs terraform_outputs.sh to extract outputs to env_vars.sh

Command Reference

Usage: deploy.sh --instance-type <ml.X.Y> --infra <cfn|tf> [OPTIONS]

Required:
  --instance-type <type>    SageMaker instance type (e.g. ml.g5.8xlarge)
  --infra <cfn|tf>          Infrastructure deployment method

Optional:
  --instance-count <N>      Number of accelerated instances (default: 4)
  --region <region>         AWS region (default: us-west-2)
  --az-id <az-id>           Availability zone ID (default: usw2-az2)
  --stack-name <name>       CFN stack name (default: hp-eks-slinky-stack)
  --help                    Show help

Verification

Deployment is successful when:

• deploy.sh exits with code 0
• env_vars.sh exists and contains all 7 exported variables
• kubectl get nodes shows nodes in Ready state
• kubectl cluster-info returns the EKS API server endpoint

# Quick verification
source env_vars.sh
echo "Cluster: $EKS_CLUSTER_NAME"
echo "Region: $AWS_REGION"
kubectl get nodes -o wide

Troubleshooting

References

• deploy.sh -- Main infrastructure deployment script
• lib/deploy_helpers.sh -- Helper functions (resolve_instance_profile, resolve_cfn_params, resolve_tf_vars, validate_az_id, validate_cfn_template, check_command)
• params.json -- CloudFormation parameters (40 params, g5 defaults)
• custom.tfvars -- Terraform variables (g5 defaults)

Source: awslabs/awsome-distributed-ai — distributed by TomeVault.