Results for “model-inference”

Train, evaluate, export, quantize, and run inference on Mask2Former models for panoptic, instance, and semantic segmentation using NVIDIA TAO.

Trains, evaluates, exports, and runs inference on FoundationStereo models for stereo depth estimation and 3D reconstruction from stereo image pairs.

Deploy and serve AI/ML models on Modal's serverless cloud platform with on-demand GPUs, autoscaling containers, persistent storage, and scheduled jobs.

Train large-scale MoE models with FP8/INT4 low-precision RL, speculative decoding, and train-inference alignment using the miles framework.

Implements the NOWAIT technique for efficient reasoning in R1-style LLMs. Use when optimizing inference of reasoning models (QwQ, DeepSeek-R1, Phi4-Reasoning, Qwen3, Kimi-VL, QvQ), reducing chain-of-thought token usage by 27-51% while preserving accuracy. Triggers on "optimize reasoning", "reduce thinking tokens", "efficient inference", "suppress reflection tokens", or when working with verbose CoT outputs.

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

Use for writing, reviewing, debugging, or testing Python analysis of Bayesian inference results with ArviZ, including 1.x DataTree groups, legacy InferenceData inputs, xarray dimensions and coordinates, conversion, summaries, R-hat/ESS/MCSE diagnostics, posterior predictive checks, PSIS-LOO, Pareto-k, and model comparison. Trigger on chain/draw shape errors, mislabeled groups, flattened samples, missing log likelihood, or misleading diagnostic claims. Do not use to construct or sample PyMC, NumPyro, or Bambi models, for generic plotting, or for deterministic statistics without Bayesian draws.

Implements the NOWAIT technique for efficient reasoning in R1-style LLMs. Use when optimizing inference of reasoning models (QwQ, DeepSeek-R1, Phi4-Reasoning, Qwen3, Kimi-VL, QvQ), reducing chain-of-thought token usage by 27-51% while preserving accuracy. Triggers on "optimize reasoning", "reduce thinking tokens", "efficient inference", "suppress reflection tokens", or when working with verbose CoT outputs.

Provides deep generative models for single-cell omics analysis, including probabilistic batch correction, transfer learning, differential expression, and multi-modal integration.

Run GGUF models locally with llama.cpp, including finding the right file on the Hugging Face Hub, installing, quantizing, serving, and using Python bindings.

Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.

Runs chest X-ray reasoning smoke tests using the NV-Reason-CXR-3B model via local inference or a public Hugging Face Space API.

Trains, evaluates, exports, and runs inference for Siamese-network-based optical inspection models to detect manufacturing defects and quality issues in image pairs.

Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.

Use for writing, reviewing, debugging, testing, or diagnosing Python Bayesian models built directly with PyMC, including Model, coords/dims, Data, random variables, potentials, posterior sampling, prior/posterior predictive checks, and InferenceData output. Trigger on model geometry, shape errors, divergences, sampler choice, mutable prediction data, and probabilistic validation. Do not use for Bambi formula models, NumPyro/JAX programs, ArviZ-only analysis of existing draws, deterministic optimization, or general statistics without PyMC code.

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

Train, evaluate, export, quantize, and run inference for a TAO OneFormer model that performs panoptic, instance, and semantic segmentation using task-conditioned queries.

Train, evaluate, export, and run inference for Masked Auto-Encoder (MAE) models for self-supervised pretraining and fine-tuning of visual representations.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.

Detect anomalies in time-series data using the Azure AI Anomaly Detector SDK for Java, with support for univariate and multivariate analysis, model training, and inference.

Trains, evaluates, exports, quantizes, and runs inference for a Grounding DINO model that detects objects described by text prompts without a fixed class vocabulary.

Fine-tune, evaluate, run inference, and export Cosmos-Embed1 video-text embedding models for tasks like text-to-video retrieval and semantic deduplication.

Run ML workloads on Modal's serverless GPU cloud: deploy models as auto-scaling APIs, run batch jobs, and schedule tasks with pay-per-second GPU pricing.

Trains, evaluates, exports, and runs inference for NVPanoptix3D models that perform panoptic 3D scene reconstruction from posed RGB images, producing 3D panoptic segmentation with occupancy completion.

Run 250+ AI apps from the command line: generate images and videos, call LLMs, search the web, create 3D models, and automate Twitter posts.

Trains, evaluates, exports, prunes, quantizes, retrains, and runs inference for TAO OCRNet models for scene text recognition from cropped text-region images, supporting CTC and attention-based decoders.

Train, evaluate, export, and run inference for monocular depth estimation models using Metric Depth Anything v2 or Relative Depth Anything architectures via the TAO toolkit.

Train, evaluate, export, and run inference on TAO action-recognition models for classifying temporal actions in video clips using RGB, optical flow, or joint input.

Trains, evaluates, and runs inference for Mask Auto-Label (MAL) weakly-supervised segmentation models using ViT-MAE backbones with minimal point or box annotations.

Trains, evaluates, exports, and runs inference for Visual ChangeNet models used in AOI defect detection, comparing image pairs for PASS/NO_PASS classification or change-segmentation masks.

Discovers and uses scientific datasets, models, blog posts, and interactive demos from a curated catalog for AI/ML work in domains like biology, chemistry, physics, and genomics.

Learn best practices for writing effective AI video generation prompts, covering shot types, camera movements, lighting, style keywords, and model-specific tips for Veo, Seedance, Wan, Grok, and others.

RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.