Results for “dpo”

Provides expert guidance for fine-tuning LLMs with Axolotl, covering YAML configs, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, and multimodal support.

Train language models with SimPO, a reference-free preference optimization method that outperforms DPO without needing a reference model.

Fine-tune and align language models using reinforcement learning with TRL, including SFT, DPO, PPO, GRPO, and reward model training.

Fine-tune models on Azure AI Foundry using SFT, DPO, or RFT, covering dataset preparation, training job submission, deployment, and evaluation.

Train large language models (7B-70B+) with RLHF using PPO, GRPO, DPO, and other algorithms, accelerated by Ray and vLLM for distributed multi-GPU setups.

Trains LLMs with SimPO, a reference-free preference optimization method that outperforms DPO, using configurable hyperparameters and workflows for various models and tasks.

Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning) with support for SFT, DPO, GRPO, KTO, RLOO, and reward model training via CLI commands.

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Discover and use DFlow documentation, Agent CLI, Trading API, Metadata API, Proof KYC, prediction markets, and the hosted DFlow docs MCP. Use before implementing DFlow features or when field-level endpoint details are needed.

Solves single- and multi-objective optimization problems with NSGA-II/III, MOEA/D, and other evolutionary algorithms, including constraint handling, Pareto front analysis, and benchmark problems.

Agent D2 - Data Collection Specialist - Interviews, Focus Groups & Observation. Covers protocol development, question design, probing strategies, transcription conventions, and systematic observation. Absorbed D3 (Observation Protocol Designer) capabilities.

Build complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framework for systematic LM programming

Fine-tune Gemma models locally using QLoRA, Unsloth, or TRL for SFT, DPO, and reward modeling, with dataset preparation and conversion to GGUF or LiteRT-LM.

Train or fine-tune language and vision models using TRL or Unsloth on Hugging Face Jobs cloud infrastructure, with support for SFT, DPO, GRPO, and reward modeling, plus GGUF conversion for local deployment.

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

Drive Soup (`soup-cli`), a CLI-first tool for fine-tuning and post-training LLMs with one YAML config and one command — SFT, DPO/GRPO/ORPO/SimPO/KTO, QLoRA/DoRA/LoRA+, layer streaming for 4-8 GB GPUs, eval-gated training, and serving. Use when the user wants to `soup init`/`soup train` a model, pick a training method or quantization scheme, estimate cost/memory before training, fine-tune on a small local GPU, migrate a config from Axolotl/LLaMA-Factory/Unsloth, or serve/merge/push a trained adapter. Triggers on: "soup-cli", "soup train", "soup init", "fine-tune an LLM locally", "QLoRA on a laptop GPU", "layer streaming", "soup advise", "soup autopilot", "DPO/GRPO/ORPO training", "merge LoRA adapter".