Results for “transformer”

Extend context windows of transformer models using RoPE, YaRN, ALiBi, and position interpolation techniques for processing long documents and implementing efficient positional encodings.

Train, evaluate, export, distill, quantize, or run inference for a TAO DINO 2D object detector using transformer-based detection with denoising training and multi-scale features.

Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.

Trains vision transformers via self-distillation without labels for self-supervised visual representation learning, and supports export and inference of NVDINOv2 backbones.

Trains and fine-tunes vision models for object detection, image classification, and segmentation using Hugging Face Transformers on cloud GPUs, with automatic dataset validation and Hub persistence.

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.

Generate high-quality sentence and text embeddings for semantic similarity, clustering, and retrieval using 5000+ pre-trained models. Supports multilingual and domain-specific embeddings for RAG and semantic search.

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Generates Python code to batch-translate the English column of tab-separated CSV files while preserving the original Chinese column and output format.

Fine-tune LLMs using LoRA, QLoRA, and PEFT with Hugging Face, including dataset preparation, hyperparameter tuning, evaluation, and deployment.

Track ML experiments with open-source run logging, local or self-hosted dashboards, and media visualization using SwanLab.

Train and run Mamba state-space models with O(n) complexity, achieving faster inference and longer context than Transformers.

Train and experiment with a minimal GPT implementation in ~300 lines of PyTorch, from character-level Shakespeare to GPT-2 scale.

Fine-tune and align language models using reinforcement learning with TRL, including SFT, DPO, PPO, GRPO, and reward model training.

Train and analyze Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features for mechanistic interpretability research.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.

Train, evaluate, distill, quantize, export, and run inference for RT-DETR object detection models using NVIDIA TAO.

Trains, evaluates, exports, quantizes, and runs inference for SegFormer semantic segmentation models using NVIDIA TAO.

Train, evaluate, export, quantize, and run inference on Mask2Former models for panoptic, instance, and semantic segmentation using NVIDIA TAO.

Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.

Benchmarks a text-to-text transformer across GLUE, SuperGLUE, CNN/Daily Mail, SQuAD, and WMT, reporting GLUE average, BLEU, ROUGE-2-F, and Exact Match scores.

Fine-tune large language models by training less than 1% of parameters using LoRA, QLoRA, and 25+ adapter methods, enabling efficient adaptation on limited GPU memory.

Train, evaluate, export, prune, and run inference for PointPillars 3D object detection models from LiDAR point clouds using NVIDIA TAO.

Selects the right Gemma model for a task, recommends deployment tooling (Gradio, Transformers.js, Vertex AI, MLX), and applies optimizations like MTP and QAT.

Quantize large language models to 8/4/3/2/1-bit precision without calibration data, using multiple optimized backends and integrations with HuggingFace Transformers, vLLM, and PEFT/LoRA.

Fine-tune large language models by training less than 1% of parameters using LoRA, QLoRA, and 25+ adapter methods, enabling efficient adaptation on consumer GPUs.

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining, covering SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

Train AI models to be harmless through self-critique and AI feedback using a set of constitutional principles, without requiring human labels for harmful outputs.

Quantize large language models to 8/4/3/2/1-bit precision without calibration data, using multiple optimized backends for deployment with vLLM or HuggingFace Transformers.

Build clinical deep-learning pipelines with PyHealth: load EHR, signal, and imaging datasets, define prediction tasks, instantiate models, train with the PyHealth Trainer, and compute clinical metrics.