Results for “model-inference”

Build AI applications with the inference.sh Python SDK: run apps, build agents, and integrate with 250+ models using sync/async, streaming, file uploads, and a tool builder API.

Generate and edit images using OpenAI's GPT-Image-2 model via the inference.sh CLI, supporting text-to-image, image editing, inpainting, and batch generation.

Run Fireworks-hosted open-weight model councils that compare responses and synthesize a final answer.

Generate and edit images using Alibaba Qwen-Image-2.0 models via the inference.sh CLI, with support for text-to-image, multi-image editing, and text rendering.

Assist with Colibri: pure-C LLM inference engine for running GLM-5.2 (744B MoE) on consumer machines with ~25 GB RAM. Use when setting up, building, converting models, running inference, configuring expert streaming and caching, optimizing speculative decoding (MTP), GPU integration, and integrating Colibri into production pipelines. Includes build setup, model download & conversion, chat/inference modes, performance tuning, and API integration patterns.

Expert en inference llama.cpp avancé (GGUF, quantization, local models, HTTP server, hardware)

AI/ML security playbook. Use when assessing model supply chain attacks (pickle RCE, poisoned weights), adversarial examples, model poisoning, model stealing, data privacy attacks (membership inference, model inversion), and autonomous agent security risks.

Generate and edit videos using Alibaba HappyHorse 1.0 models via the inference.sh CLI, supporting text-to-video, image-to-video, reference-to-video, and video editing with natural language.

Quantize large language models to 4-bit precision using activation-aware weight quantization, reducing memory footprint and speeding up inference with minimal accuracy loss.

Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.

Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.

Integrate 250+ AI models into JavaScript/TypeScript apps with streaming, file uploads, and agent tooling.

Generate videos using Pruna's optimized P-Video and WAN models via the inference.sh CLI, supporting text-to-video, image-to-video, audio input, and multiple resolutions.

Build, fit, validate, and compare Bayesian models using PyMC's modern API, including hierarchical models, MCMC sampling, variational inference, posterior predictive checks, and model comparison.

Generate high-quality speech from text using ElevenLabs' premium voices, with support for 32 languages, multiple models, and voice tuning parameters.

Estimates the memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub, using HTTP Range requests without downloading weights.

Generate natural AI voices, text-to-speech, and voice synthesis using the inference.sh CLI with models like Inworld TTS, ElevenLabs, and Kokoro TTS for voiceovers, audiobooks, podcasts, and more.

Fit statistical models (OLS, GLM, ARIMA, mixed models) with detailed diagnostics, residuals, and inference for econometrics and time series analysis.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.

Estimates GPU memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub using HTTP Range requests, without downloading weights locally.

Learn and apply prompt engineering techniques for LLMs, image generators, and video models using the inference.sh CLI.

Simulates meiotic recombination to produce offspring genomes from parent pairs, modeling Mendelian segregation, de novo mutation, sex determination, trait inference, and clinical evaluation against a disease registry.

Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.

Train, evaluate, distill, quantize, export, and run inference for PyTorch-based TAO image classification models with support for multiple backbones.

Python SDK for inference.sh - run AI apps, build agents, and integrate with 150+ models. Package: inferencesh (pip install inferencesh). Supports sync/async, streaming, file uploads. Build agents with template or ad-hoc patterns, tool builder API, skills, and human approval. Use for: Python integration, AI apps, agent development, RAG pipelines, automation. Triggers: python sdk, inferencesh, pip install, python api, python client, async inference, python agent, tool builder python, programmatic ai, python integration, sdk python

Transcribe audio with high accuracy using ElevenLabs Scribe models, supporting speaker diarization, audio event tagging, forced alignment, and subtitle generation via the inference.sh CLI.

Fine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot policy inference across ALOHA, DROID, and LIBERO environments. Use when adapting pi0 models to custom datasets, converting JAX checkpoints to PyTorch, running policy inference servers, or debugging norm stats and GPU memory issues.

Create professional marketing videos for ads, promos, product launches, and brand content using AI video generation models and voiceover tools via the inference.sh CLI.

Trains, evaluates, exports, and runs inference for person re-identification models using TAO, learning discriminative embeddings for cross-camera matching.

Operate Hugging Face Hub repositories, models, datasets, and Spaces via the hf CLI, including downloads, uploads, authentication, and compute jobs.

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

Deposit ERC-20 token supply into a Berry Juicer vault on Base to earn trading fees as USDC, then spend the yield as AI inference across 140+ models.

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.