adu2021
- 1.2k skills
- 0 followers
- 1 day ago last updated
- ▌ First Frame Video Customization · adu2021Enable video generation customization via first-frame reuse—treat initial frame as a visual entity buffer storing objects for reuse during generation with just 20-50 examples, requiring minimal architectural change.
- ▌ Flex Continuous Agent Evolution · adu2021Enable LLM agents to improve continuously during deployment by constructing structured experience libraries through self-reflection on successes and failures—achieving 23% improvement on reasoning without gradient-based parameter updates or external training.
- ▌ Fp32 Reproducible LLM Inference · adu2021Diagnose and solve LLM reproducibility failures caused by floating-point precision across hardware configurations using LayerCast optimization for deterministic inference with minimal memory overhead.
- ▌ Glance Phase Aware Acceleration · adu2021Phase-aware acceleration using two lightweight LoRA adapters (Slow-LoRA for semantic reconstruction, Fast-LoRA for texture refinement) trained on a single image in one GPU hour, achieving 5× speedup via smart per-phase acceleration rather than uniform speedup.
- ▌ High Entropy Minority Tokens Rl · adu2021Optimize only high-entropy tokens during RL training to achieve better reasoning performance with 80% fewer gradient updates.
- ▌ Latent Collaboration Multiagent · adu2021Enable LLM agents to collaborate entirely in latent space by sharing layer-wise KV cache representations instead of text, achieving 4× speedup and 71-84% fewer output tokens while maintaining 14.6% higher accuracy through lossless information preservation in continuous embeddings.
- ▌ Layer Cake Contrastive Decoding · adu2021Improve factual accuracy in LLM generation through decoding-time layer-wise attention suppression. Selectively suppress attention to specific token types at their most influential transformer depths without training or model modifications. Use when you need to reduce hallucinations and improve factual correctness at inference time.
- ▌ Length Incentivized Exploration · adu2021Overcome the shallow exploration trap by explicitly rewarding longer reasoning sequences when models fail to solve problems. Use length-incentivized exploration to enable deeper chain-of-thought reasoning, achieving better test-time scaling and improved generalization across in-domain and out-of-domain tasks.
- ▌ Llm2vec Gen Response Embeddings · adu2021Learn to encode LLM-generated responses rather than raw inputs by training special tokens and projection layers while keeping the backbone frozen. Bridges the input-output gap to transfer LLM capabilities like reasoning directly into embedding space.
- ▌ Longvila Scaling Rl Long Videos · adu2021Train vision-language models on hour-long videos using a two-stage pipeline combining supervised fine-tuning with GRPO, reaching 71% accuracy on VideoMME while supporting 8,192 frames through efficient multi-modal sequence parallelism.
- ▌ Luspo Length Unbiased Sequences · adu2021Fix length bias in sequence-level policy optimization by scaling each sequence's loss by its token count, eliminating gradient imbalances that cause models to shorten responses during training.
- ▌ Lychee Decode Sparse Kv Sharing · adu2021Classify attention heads into retrieval (full attention) and sparse (token-selected) roles using HardKuma distribution for differentiable discrete optimization. Sparse heads reuse KV pairs from retrieval heads, reducing cache by 90% while maintaining quality through joint training.
- ▌ Mathse Iterative Math Reasoning · adu2021Improve multimodal mathematical reasoning through iterative reflection cycles where an outcome reward model provides feedback on reasoning quality, and correct solutions are incorporated back into training—enabling continuous model adaptation beyond static datasets.
- ▌ Olympiad Long Horizon Reasoning · adu2021Solve IMO-level problems through multi-stage hierarchical reasoning with lemma-based memory compression. Intern-S1-MO achieves 26/35 on IMO2025 geometry-excluded problems—ideal when complex reasoning exceeds single-pass context.
- ▌ On Policy Reasoning Compression · adu2021Compress verbose reasoning by conditioning the same model on a conciseness instruction to create a teacher. Minimize KL divergence between student outputs and concise teacher without ground truth, enabling implicit reward learning that improves accuracy while reducing tokens.
- ▌ Qwenlong Long Context Reasoning · adu2021Systematically post-train models for long-context reasoning through multi-hop data synthesis, stabilized RL with adaptive entropy control, and memory-augmented architecture supporting 4M+ token sequences. Achieves performance comparable to GPT-5 and Gemini-2.5-Pro on long-context benchmarks.
- ▌ Redsearcher Long Horizon Agents · adu2021Train language models for multi-step information-seeking using dual-constrained task synthesis and cost-efficient staged learning. Generate complex queries by controlling topological complexity and information dispersion, then train atomic reasoning skills before introducing environment interaction. Achieve state-of-the-art on long-horizon search benchmarks with 30B parameter models.
- ▌ Rwml Reinforcement World Models · adu2021Train LLM agents to anticipate environment consequences by learning world models through reinforcement learning with embedding-space similarity rewards, avoiding task-specific labels while enabling robust environment adaptation.
- ▌ Skill Rl Recursive Distillation · adu2021Improve agent performance by autonomously distilling behavioral patterns from trajectories into reusable skills, then using these skills to guide future decisions. Achieves 89.9% success on ALFWorld through differential processing of success vs failure episodes and dynamic skill library evolution.
- ▌ Sparse Moe Agentic Intelligence · adu2021Deploy frontier-level reasoning with only 11B active parameters using sparse MoE with 288 routed experts plus shared expert. Use Metropolis Independence Sampling-Filtered Policy Optimization (MIS-PO) to stabilize RL training at scale, replacing continuous importance weighting with discrete filtering that ensures trust region stability.
- ▌ Tarot Test Driven Curriculum Rl · adu2021Design capability-adaptive curricula for code generation by constructing per-problem test suites (basic, intermediate, complex, edge cases) and dynamically weighting training signals based on model capability rather than problem difficulty alone. Optimal curricula vary by model capacity: weaker models benefit from easy-to-hard progression while stronger models learn faster with complex-first strategies, enabling 1 dataset to serve multiple model scales efficiently.
- ▌ Terminal Bench Agent Evaluation · adu2021Evaluate agents on 89 challenging terminal-based tasks where frontier models score below 65%, providing realistic assessment of command-line interface automation capability. Use when benchmarking agents designed for system administration, automation, or DevOps tasks.
- ▌ Text Video Alignment Evaluation · adu2021Evaluate text-to-video alignment through fine-grained semantic understanding via multi-agent question generation and knowledge-augmented answering. Generate 12,000 atomic yes/no questions from 2,000 prompts across 10 evaluation categories, achieving 58.47 correlation with human judgment.
- ▌ Threadweaver Parallel Reasoning · adu2021Enable parallel reasoning threads on standard autoregressive inference engines without modifications to position embeddings or KV cache. ThreadWeaver achieves 1.53× speedup while maintaining chain-of-thought quality—ideal when you need faster reasoning without special hardware.
- ▌ Tir Bench Visual Tool Reasoning · adu2021Evaluate multimodal models on agentic visual reasoning through 13 diverse tasks requiring novel tool use for image processing and manipulation in chain-of-thought, revealing that strong performance requires genuine thinking-with-images capabilities.
- ▌ Token Bottleneck Scene Dynamics · adu2021Learn to compress entire scenes into a single bottleneck token that captures temporal dynamics. Enables efficient visual tracking and robotic manipulation by forcing reconstruction from minimal target hints, achieving superior performance with training costs comparable to standard autoencoders.
- ▌ Toucan MCP Tool Agentic Dataset · adu2021Generate 1.5M realistic agent training trajectories from 495 real-world MCP servers without human annotation. Use to build tool-agentic training datasets with authentic tool execution, multi-turn conversations, and error handling.
- ▌ Two Grpo Contrastive Efficiency · adu2021Reduce GRPO training cost by 87.5% using only 2 rollouts instead of 16 while achieving 98.1% of baseline performance. Leverage the insight that GRPO's group mechanism serves contrastive learning rather than advantage estimation.