# Hugging Face On Foundry > Deploy a model from the Hugging Face Hub to Microsoft Foundry on Managed Compute. - Skill: `alvarobartt/hugging-face-on-foundry` (Agent Skill, multi-file: 8 files) - Install (CLI): `npx skillmds@latest add alvarobartt/hugging-face-on-foundry` - Raw SKILL.md: https://api.skillmd.com/api/skills/alvarobartt/hugging-face-on-foundry/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: alvarobartt (https://skillmd.com/u/alvarobartt) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/alvarobartt/hugging-face-on-foundry --- Deploy an open-model from the Hugging Face Hub to Microsoft Foundry using Managed Compute with both the `az` CLI and the Python SDK. ## When to use Use this skill when the user asks things like: - "Can you deploy Qwen3.6-27B-FP8 on Foundry?" - "Deploy this model on Azure Foundry with Managed Compute" - "Help me deploy and test a model in Foundry" Do not use it for Serverless API deployments, as those fall under a different category. ## Prerequisites - An Azure subscription with access to Microsoft Foundry - A Microsoft Foundry resource - `az` CLI installed, be logged in with `az login`, and `ml` extension installed - Python 3.10 or higher installed - `pip install azure-identity "azure-ai-ml>=1.34.0" "azure-mgmt-cognitiveservices>=15.0.0b2" huggingface_hub openai` It's recommended when applicable to use `uv` with an environment, and install Python packages with `uv pip install ...` instead of `pip install ...`. So unless there's already an active environment in `.venv`, then create it as `uv venv --python 3.11` and activate it as `source .venv/bin/activate` for Bash. If something is not installed, suggest the user to install or update / upgrade those, rather than moving on without any of those installed. Also note that both commands as well as the reference snippets are using mock values surrounded with <> e.g., , which will need to be replaced with the actual values. When you ask the user something, make sure that's formatted accordingly so that the user only needs to select one of the available options, or rather reply with free-form text when applicable. And ask as many questions as required, and do not collapse choices into vague options like "this resource or another one from the subscription". At minimum, the user must explicitly choose the target Foundry Account / Project and the deployment-template to use before you deploy. Do not choose either of those on behalf of the user unless they explicitly instructed you to proceed without further questions. ## Step-by-step Go through the following steps one-by-one, and make sure that the user confirms the decision before moving onto the next step. ### 0. Question Initially, you will need to ask the following questions, unless those things have already been provided: - Could you provide the Hugging Face Hub ID of the model you want to deploy? - Which Azure subscription should be used? - Feel free to check the default in `az account show -o tsv --query id` - Note that the `SUBSCRIPTION_ID` will be required later on - In which Foundry Account / Project you want to deploy the model? - For this, prefer a concise listing over raw JSON, e.g. `az cognitiveservices account list --subscription --query "[].{name:name,resourceGroup:resourceGroup,location:location,kind:kind,customSubDomainName:properties.customSubDomainName}" -o table` - Note that the `ACCOUNT_NAME` i.e., Foundry project name, will be required later on With the information pulled via the `az` CLI, always ask the user for confirmation before sticking to a resource or the other, and make that confirmation a choice among the resources found, letting the user decide which one to use, unless it's been provided beforehand. ### 1. Validate The user should've provided a model ID from the Hugging Face Hub, which you will need to either complete or verify. The model ID might've been provided as `Qwen3.6-27B`, which is not correct, since it's missing the author information, in this case being `Qwen`, so the complete model ID is `Qwen/Qwen3.6-27B`. For that, you should use the `hf` CLI to search for whatever the user prompts as `hf models list --search "Qwen3.6 27B" --sort downloads --format json --limit 10` and then get the value of those results by `id` to see which one is the user referring too. Additionally, you can as well run the same query but sorting by trendiness as `hf models list --search "Qwen3.6 27B" --sort trending_score --format json --limit 10`. If the model is indeed provided completely as `Qwen/Qwen3.6-27B`, you still need to verify that's a public and non-gated model that's available on the Hugging Face Hub with `hf models info Qwen/Qwen3.6-27B`, which can either return the JSON with the model information or rather an error like "Error: Model '...' not found.". Note that if the JSON has been returned, you will need to ensure that `private: false` and `gated: false`. ### 2. Find After the model is validated, you should also check that the model exists on Microsoft Foundry, so for that you need to use the `references/foundry-model.py` so that given the Hugging Face Hub model ID, you run it to check whether the model is available, and if so, which is it's latest version. If not available, consider the model not deployable for the moment, as they will need to reach out to Hugging Face to onboard that model. Assuming that the provided model was clear enough and is available on the Hugging Face catalog on Microsoft Foundry i.e., the `azure-huggingface` registry, then you need to run `references/foundry-deployment-templates.py` to get the latest version of the current model, as well as the deployment-templates it supports and their information such as the `acceleratorType` and the `description`, to help the user take a decision later on. Note that the specific version can be ignored as Foundry will always use `/labels/latest`, but it's required to pull the information for a given deployment-template. If the SDK shape differs from the reference script, inspect `model._to_dict()` and look for `default_deployment_template.asset_id`. Besides that, note that as Microsoft Foundry relies on a global managed quota, you need to use `references/foundry-capacity.py` to check the available capacity on a given project to see how many instances for NVIDIA A100, NVIDIA H100 or AMD MI300x are available to use. ### 3. Deploy Then with the model and the deployment-template URIs you need to select one deployment-template to use, which can come in multiple flavours depending on the capabilities as e.g., context-length, or the instance/s it uses. Note that the only available `acceleratorType` values as of today are A100_80GB, H100_80GB, and MI300_192GB, and their cost is $3.95, $7.91, and $7.91, per accelerator, respectively, so a deployment-template that requires 4 accelerators, its cost will be 4 times that per hour. Given that running those costs money, you should ask the user which one of the different alternatives they want to deploy, including the trade-off between those if any, otherwise simply mention the difference on the price, unless there's something in each deployment-template description that's relevant for the user to know. Present each valid deployment-template as an explicit option, not as prose that implies a default unless the user objects. Summarize each option using only fields and capabilities that are actually available for that deployment-template. For example, include accelerator type, accelerator count, estimated cost, default-template status, context length, environment variables, runtime tuning, or task/modality only when those values are present or can be inferred directly from the template metadata. If a field is missing, omit it instead of guessing. If there is only one valid deployment-template, still ask the user to confirm that template together with the target Foundry resource and the billable accelerator cost. If they rather prefer you to go ahead without asking them, then use whichever is the default deployment-template, and if it is either CPU or NVIDIA T4 i.e., not supported on Microsoft Foundry, then out of the existing deployment-templates use the cheapest or most efficient in the following order: A100_80GB, MI300_192GB, and H100_80GB. Choose a deployment name derived from the model ID, e.g. `qwen36-27b-fp8`; it must be 2 to 64 characters and only include alphanumeric characters, underscores, and hyphens. Once they decide, then simply run `references/foundry-deploy.py` with the actual replacements and wait until the endpoint is running on Microsoft Foundry. After deployment, capture the result and note `properties.routes.chatCompletionsScoringPath` when present, since it is useful for debugging even though the preferred user-facing route is the Foundry router. Additionally, to track the deployment progress since it might take ~15-30 minutes (or more depending on the availability), you can use the `reference/foundry-poll.py` to check the status of the deployment, and poll until "Succeeded". ### 4. Use Once deployed, you will be able to use it i.e., send HTTP requests to it. As it's deployed under a router on Microsoft Foundry, you need to send OpenAI-compatible requests to e.g., Chat Completions API via an endpoint that looks like `https://.services.ai.azure.com/openai/v1`. Then you have `references/foundry-primary-key.py` on how to obtain the primary key to be used to send requests to the endpoint. Note that for chat-completion models that usually means either `/v1/chat/completions` or `/v1/responses` (for reasoning models). ```bash curl https://.services.ai.azure.com/openai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $(az cognitiveservices account keys list --subscription --resource-group --name -o tsv --query key1)" \ -d '{"model":"","messages":[{"role":"user","content":"How does Hugging Face make money?"}],"stream":true}' ``` Alternatively, you can also run `references/foundry-chat.py` for the chat completions models to send requests to the Chat Completions API via the OpenAI SDK, but note that you can also use the Responses API if the model supports reasoning, since it's better supported with Responses API. ## Troubleshooting As all the models in the `azure-huggingface` catalog go through an extensive testing process those shouldn't fail on deployment time, but if those fail on inference time make sure to report those. If otherwise, those do fail on deployment time, note that the issue might come due to provisioning or allocating the instances. ## References - ["/goal GLM 5.2 on Microsoft Foundry with AMD MI300"](https://alvarobartt.com/goal-glm-5.2-on-foundry)