adu2021
- 1.2k skills
- 0 followers
- 1 day ago last updated
- ▌ Margin Adaptive Dpo · adu2021Adapt DPO temperature parameters per sample using reward model margins, amplifying learning signals for hard preference pairs while dampening easy ones.
- ▌ Nemotron Cascade Rl · adu2021Train language models through sequential, domain-wise RL stages (RLHF → Instruction-Following → Math → Code → SWE) without catastrophic forgetting. Exploit policy-dependent training data distribution where previous behaviors persist when reward-relevant. 14B model surpasses DeepSeek-R1-0528 (671B) on LiveCodeBench.
- ▌ Prefix Grouper Grpo · adu2021Implement Prefix Grouper to accelerate Group Relative Policy Optimization training by eliminating redundant prefix encoding, achieving up to 8x speedup for long-context scenarios.
- ▌ Qwen3 Vl Multimodal · adu2021State-of-the-art multimodal model advancing vision-language understanding and generation capabilities through improved visual encoders, dense token representations, and unified reasoning over images and text.
- ▌ Recurrent Depth Vla · adu2021Enable test-time compute scaling in vision-language-action models via weight-tied recurrent inference within latent space, with adaptive stopping based on action divergence.
- ▌ Rl Anything Dynamic · adu2021Enable simultaneous optimization of environment difficulty, policy, and reward model. System uses reward model evaluations to guide environment adaptation, creating positive feedback loop for scalable agent improvement.
- ▌ Rl Visual Reasoning · adu2021RL in vision-language models improves inference-time alignment between vision and reasoning in mid-to-late layers, not vision ability or reasoning separately. Strategic layer freezing enables diagnosis of which components contribute to RL gains.
- ▌ Search R3 Reasoning · adu2021Unify LLM reasoning with embedding generation by training models to output embeddings as direct outputs of step-by-step reasoning. Trigger: improve performance on complex retrieval tasks requiring deep reasoning.
- ▌ Spatial Forcing Vla · adu2021Align intermediate visual embeddings in vision-language-action models with 3D geometric representations from pretrained foundation models. Improves spatial understanding and enables faster training (3.8x speedup) without explicit 3D inputs.
- ▌ Spatiallm 3d Scenes · adu2021Train multimodal LLMs to parse 3D point clouds and generate executable Python code representing structured indoor scene layouts with walls, doors, and objects.
- ▌ Sr Grpo Stable Rank · adu2021Uses stable rank (variance distribution across semantic dimensions) as annotation-free reward signal in GRPO to incentivize high-dimensional representation structures, eliminating dependency on human preference annotations or learned reward models.
- ▌ Srum Self Rewarding · adu2021Enable unified multimodal models to self-improve by using their understanding module as evaluator for generation. Apply hierarchical dual rewards: global for overall semantics and local for fine-grained details.
- ▌ Ssrl Self Search Rl · adu2021Enable LLMs to perform internal knowledge search using structured prompting and rule-based rewards, reducing reliance on external search while maintaining accuracy and reducing hallucination.
- ▌ Stable Rl With Llms · adu2021Justifies token-level optimization objectives as first-order approximation to sequence rewards via Routing Replay, which fixes MoE routers during policy optimization to reduce training-inference discrepancy. Use when scaling RL training on large MoE language models.
- ▌ Todoevolve Planning · adu2021Autonomously synthesize task-specific agent planning architectures by optimizing for correctness, stability, and efficiency via impedance-guided preference optimization.
- ▌ Uniqueness Aware Rl · adu2021Improves LLM reasoning by rewarding correct solutions that exhibit rare high-level strategies, preventing exploration collapse and discovering more diverse solution approaches across mathematics, physics, and medical reasoning.
- ▌ Videossm Long Video · adu2021Generate minute-scale coherent videos using state-space models as evolving memory for scene dynamics. VideoSSM achieves linear computational complexity while reducing motion drift—ideal when temporal consistency matters across long video sequences.
- ▌ Speed By Simplicity · adu2021Replace multi-stream modality-specific pathways with a unified Transformer backbone processing text, video, and audio tokens in shared sequence via self-attention. Achieves superior visual quality (4.80 vs 4.76), 75% better speech clarity (14.6% WER vs 19.23%), and 80% human preference wins—particularly strong for human-centric scenarios with expressive facial performance and audio-video sync.
- ▌ Alphapollo Reasoning · adu2021Enable LLMs to solve complex problems through multi-turn agentic reasoning with tool-assisted verification and iterative refinement loops. Trigger: improve reasoning reliability on long-horizon tasks by combining RL with verification.
- ▌ Ares Entropy Shaping · adu2021Calibrate exploration effort in reasoning traces based on problem difficulty by detecting high-entropy windows and applying hierarchical entropy rewards. Reduces unnecessary reasoning on easy tasks while increasing exploration on hard tasks.
- ▌ Bitvla Robot Control · adu2021Build fully ternary quantized vision-language-action models for robotic manipulation, achieving 11x memory reduction and 4.4x speedup while maintaining task performance on edge devices.
- ▌ Chaining Evidence Rl · adu2021Train search agents using citation-aware rubric rewards that decompose complex questions into verifiable single-hop facts. Agents learn to chain evidence through explicit source citations, preventing hallucinations and shortcut exploitation. Citation-aware Group Relative Policy Optimization (C-GRPO) combines rubric and outcome rewards, enabling agents to solve multi-hop reasoning tasks with high factual grounding and transparency.
- ▌ Compass Event Memory · adu2021Organize agent memory as an event graph with explicit logical relationships rather than flat embeddings. Framework incrementally segments experiences into events and links them through causal, temporal, and logical relations. Enables agents to navigate memory as a logic map for goal-directed searching and structured reasoning, improving performance on multi-hop reasoning and long-horizon planning tasks.
- ▌ Confucius Code Agent · adu2021Build AI code agents that scale to massive repositories with long-context reasoning and persistent memory. Confucius SDK achieves 59% Resolve@1 on SWE-Bench-Pro—ideal when AI needs to handle real-world codebases with complex toolchains.
- ▌ Critique Rl Training · adu2021Trains language models to provide quality feedback through two-stage RL. Stage 1 optimizes discriminability (distinguishing good vs bad responses). Stage 2 adds helpfulness rewards (improving actor after feedback). Achieves 9.02% improvement without requiring stronger supervisors for training data.
- ▌ Deep Agent Reasoning · adu2021Enables autonomous reasoning agents to discover and invoke tools efficiently through end-to-end training. Uses autonomous memory folding to compress interaction history and ToolPO to learn general-purpose tool use, applicable across diverse benchmarks from QA to web automation.
- ▌ Dice Cuda Generation · adu2021Train diffusion language models to generate optimized CUDA kernels using bi-phase reinforcement learning. First phase masks and regenerates core kernel logic with provided scaffolding to prevent PyTorch shortcuts. Second phase enables end-to-end generation. Leverage diffusion's global context awareness for non-sequential code generation.
- ▌ Dover Auto Debugging · adu2021Diagnose and fix multi-agent system failures through targeted interventions (message edits, plan changes) rather than static log analysis. DoVer recovers 18-28% of failed trials with 30-60% hypothesis validation—essential for autonomous multi-agent reliability.
- ▌ Hop Skip Overthink Diagnosis · adu2021Novel error categorization framework examining failures across hops (diversity), coverage, and overthinking. Combines human annotation with automated metrics to diagnose why reasoning models hallucinate on multi-step tasks.
- ▌ Hyperagents Self Improvement · adu2021Enable AI systems to recursively improve themselves by making the meta-level modification procedure itself editable, achieving open-ended capability growth.
- ▌ Interactive Video Generation · adu2021Convert pre-trained latent video diffusion into real-time autoregressive generators using adversarial post-training, achieving 24fps streaming on single H100.
- ▌ Inverse Llava Text To Vision · adu2021Map text embeddings into visual representation space for multimodal fusion, eliminating expensive image-text alignment pre-training while improving reasoning-heavy tasks by up to 27.2%.
- ▌ Latent Particle World Models · adu2021Learn world models by decomposing scenes into latent particles with per-particle dynamics. Each particle represents an object with position, scale, and appearance. Learn distributed latent actions governing per-particle transitions, enabling multimodal video generation from identical initial conditions.
- ▌ Lightweight Memory Augmented · adu2021Reduce inference tokens by up to 38× and API calls by 30× through three-stage memory system: sensory compression, short-term consolidation, and offline long-term storage.
- ▌ LLM Diverse Creative Writing · adu2021Learn to enhance LLM post-training for diverse creative outputs by weighting training pairs using deviation metrics (semantic and style diversity). Applies to models where standard alignment reduces diversity, enabling competitive quality with higher output variety.
- ▌ Lumos 1 Autoregressive Video · adu2021Generate videos autoregressively by extending LLM architectures to spatiotemporal data. MM-RoPE balances frequency spectra across temporal and spatial dimensions, while Autoregressive Diffusion Forcing enables efficient parallel decoding. Lumos-1 (0.5B-3B variants) matches or exceeds Show-o2 and COSMOS on text-to-video with training on 48 GPUs.
- ▌ M4olgen Molecular Generation · adu2021Generates molecules meeting precise numeric property constraints across multiple dimensions through two-stage multi-agent framework with fragment-level edits and Group Relative Policy Optimization, improving validity and property satisfaction.
- ▌ Mattrl Multi Agent Reasoning · adu2021Enables LLM-based agent teams to improve reasoning accuracy at inference time through collaborative deliberation and structured experience retrieval, achieving 3-8% accuracy gains without expensive multi-agent training.
- ▌ Meanflow One Step Generation · adu2021Trains mean-velocity models on rectified couplings from pretrained flow models to dramatically smooth loss landscape, enabling faster convergence and superior one-step generation quality without additional training data.
- ▌ Modular Large Model Training · adu2021Train large models efficiently across heterogeneous hardware (GPUs, TPUs, Trainium) using strict encapsulation principles, achieving constant code complexity when adding features across hundreds of modules.
- ▌ Multi Agent Memory Framework · adu2021Design multi-agent systems with brain-inspired memory mechanisms that enable efficient information sharing and coordination. Implement hierarchical memory structures (working memory, episodic memory, semantic memory) similar to neuroscience models to improve multi-agent reasoning, planning, and task completion.
- ▌ One Sample Polymath Learning · adu2021Demonstrate that a single strategically engineered training sample can improve reasoning across multiple domains. Polymath learning shows sample quality and multidisciplinary design matter more than quantity, enabling extreme data efficiency in RL training.
- ▌ Othink R1 Fast Slow Thinking · adu2021Enable reasoning models to adaptively switch between fast direct responses and slow detailed reasoning, reducing token consumption by 15-40% while maintaining accuracy through dual-mode fine-tuning.
- ▌ Poet Orthogonal LLM Training · adu2021Improve LLM training stability and generalization by reparameterizing weight matrices as orthogonal transformations, achieving better perplexity than AdamW with fewer trainable parameters.
- ▌ Psa Pyramid Sparse Attention · adu2021Replaces binary keep/drop masks with multi-level pooled key-value representations, allowing queries to access larger receptive fields under same compute budget through hierarchical aggregation without discarding information.
- ▌ Remit Rl Guided Mid Training · adu2021Improve LLM reasoning by reweighting pre-training data during mid-training based on discrepancies between RL-tuned and base models, boosting reasoning performance without external teachers or extra data.
- ▌ Rlp Rl Pretraining Objective · adu2021Integrate reinforcement learning into the pretraining phase by measuring the utility of intermediate reasoning for predicting subsequent tokens. This approach generates dense reward signals during standard pretraining, enabling models to develop reasoning abilities earlier and with significant performance gains (19% improvement on 1.7B, 45% lift on 12B models).
- ▌ Ruscarl Rubric Scaffolded Rl · adu2021Guide LLM exploration through rubric-based scaffolding that gradually diminishes, enabling models to internalize reasoning patterns while maintaining exploration quality for robust RL training.
- ▌ Safety At One Shot Lm Repair · adu2021Recover safety alignment in fine-tuned LLMs using only a single safety example, without sacrificing utility. Leverages low-rank structure of safety gradients to enable minimal-cost correction even when models have been compromised by extensive harmful training data.
- ▌ Scaler Reasoning Environment · adu2021SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning. From arXiv:2601.04809
- ▌ Sciarena Evaluation Platform · adu2021Build community-driven evaluation platforms for scientific tasks using pairwise model comparisons and human voting. Assess foundation models on literature-grounded reasoning without automated metrics.
- ▌ Step Entropy Cot Compression · adu2021Reduces chain-of-thought verbosity through step entropy metrics that identify and prune low-information reasoning steps while maintaining accuracy.
- ▌ Test Time Scaling Reflective · adu2021Scale model performance at test time by generating multiple reasoning trajectories and selecting the best using a self-supervised process reward model. MetaStone-S1 achieves 32B-equivalent performance using only 32B parameters and 53M for trajectory scoring, learning process rewards from outcome labels alone without process annotations.
- ▌ Text Aware Image Restoration · adu2021Restore degraded images while preserving textual fidelity using TeReDiff, a multi-task diffusion framework integrating text spotting with U-Net features and VLM-verified dataset curation.
- ▌ Tool R0 Self Evolving Agents · adu2021Tool-R0 trains tool-use agents through self-play between Generator and Solver, creating curriculum-like progression without manual task curation.
- ▌ Training X Ray Vision Amodal · adu2021Infer complete object structure despite occlusion using multi-camera video. Enables training models to predict hidden object appearance by combining temporal and spatial context from multiple viewpoints.
- ▌ Tv2tv Interleaved Video Text · adu2021Mixture-of-Transformers jointly learning language modeling and video flow matching, enabling interleaved text-video generation where semantic decisions happen in language, pixel generation in video, and users can intervene textually at any step.
- ▌ Twinflow One Step Generation · adu2021Train single-step image generators without teacher models or standard adversarial networks. Achieves 0.83 GenEval score at 1-NFE with 100× computational efficiency gains—when you need real-time image synthesis from pre-trained diffusion models.
- ▌ Unmasking Diffusion Policies · adu2021Learn which tokens to unmask during diffusion sampling via reinforcement learning instead of heuristics. Policies eliminate manual tuning and scale across block sizes—crucial when semi-autoregressive generation needs dynamic, learned unmasking strategies.
- ▌ Verltool Agentic Rl Tool Use · adu2021Train agents to leverage external tools across domains using VerlTool's unified RL framework. Coordinate code execution, search, SQL queries, and vision utilities in multi-turn interactions without domain-specific redesign. 2× faster asynchronous rollouts on mathematical reasoning, knowledge QA, and software engineering tasks.
- ▌ Vlingnav Embodied Navigation · adu2021VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory. From arXiv:2601.08665
- ▌ Vmoba Sparse Video Attention · adu2021Reduces video diffusion training compute 2.92× through spatio-temporal sparse attention with layer-wise cyclic block partitioning. Maintains generation quality while enabling long-sequence video training. Use when training video models on memory-constrained hardware or processing longer video sequences.
- ▌ Adaptive Speculative Decoding · adu2021Accelerate LLM inference 1.5-2x using a universal draft model that adapts to user data in real-time, handling different target models and tokenizers via online n-gram cache and hybrid distillation.
- ▌ Adversarial Code Verification · adu2021Generate adversarial test suites that catch LLM code errors by analyzing constraint differences between correct and incorrect solutions. SAGA framework improves test detection rate 9.55% and verifier accuracy 12.14% by using human bug patterns and multi-input differential analysis.
- ▌ Agent Fold Context Management · adu2021Enables web agents to handle long-horizon tasks by actively managing context workspace. Implements granular condensations of recent steps and deep consolidations of multi-step sub-tasks, preventing context saturation. Achieves 36.2% on BrowseComp with 30B model, matching larger proprietary agents.
- ▌ Agent Ocr History Compression · adu2021Compress agent interaction history by converting observation-action sequences into compact visual representations (images), leveraging visual tokens' superior information density. Implements segment optical caching with 20x rendering speedup and enables dynamic compression rates. Preserves over 95% of agent performance while reducing token consumption by 50%+, enabling agents to maintain longer interaction histories within fixed budgets.
- ▌ Attntrace Context Attribution · adu2021Identify influential texts in long contexts via attention weights using top-K filtering and context subsampling, achieving 10-20x speedup over perturbation methods.
- ▌ Autoenv Environment Synthesis · adu2021Generate diverse, validated game environments automatically using domain-specific language specifications and LLM coding agents with self-repair, enabling cost-effective (≈$4/env) creation of heterogeneous test domains for evaluating cross-environment agent generalization.
- ▌ Autonomous Research Framework · adu2021Build autonomous research agents using pre-computed knowledge graphs instead of online reasoning. Extract methodological patterns from literature, organize them into structured knowledge, and enable agents to align user research intents with established paradigms for efficient, grounded research planning and execution.
- ▌ Config Knowledge Distillation · adu2021Improve student model robustness under covariate shift by using diffusion-based augmentation that targets spurious features via teacher-student disagreement.
- ▌ Dash Faster Shampoo Optimizer · adu2021Accelerate the Shampoo optimizer 4.8x using batched block-wise preconditioning and numerical approximations, enabling more frequent preconditioner updates without computational bottleneck.
- ▌ Data Efficient Robot Learning · adu2021Train efficient robot manipulation policies by strategically applying task diversity and debiasing expert demonstrations to remove execution speed variations that degrade learning.
- ▌ Deepconf Confidence Filtering · adu2021Filter low-quality reasoning traces using model-internal confidence signals at test time, eliminating weak paths during generation to achieve 99.9% accuracy while reducing token generation by up to 84.7%.
- ▌ Dynamic Mask Sparse Attention · adu2021Sparse attention mechanism combining content-aware and position-aware sparse patterns through dynamic masking. Achieves 10x speedup while maintaining model quality on long-context benchmarks through hardware-friendly implementation.
- ▌ Embodied R1 Robotic Reasoning · adu2021Bridge vision-to-action gap using pointing as unified intermediate representation, enabling 56.2% success on manipulation tasks without task-specific fine-tuning.
- ▌ Emergent Temporal Abstraction · adu2021Discover hierarchical temporal abstractions within autoregressive models via internal RL, enabling efficient exploration of sparse-reward tasks. Metacontroller learns abstract action sequences modifying residual streams, switching gates enable quasi-binary patterns, and abstract-space RL achieves many orders-of-magnitude speedup over token-level learning.
- ▌ Entropy Exploration Reasoning · adu2021One-line code modification augmenting RL advantage function with clipped entropy term to encourage exploratory reasoning chains while maintaining optimization stability.
- ▌ Exploration Exploitation Rlvr · adu2021Investigate exploration-exploitation trade-offs in reinforcement learning with verifiable rewards through theoretical analysis and empirical validation. Derive explicit clipping bias bounds, establish policy-entropy shift formulation, and introduce reward-misalignment framework. Show policy entropy and performance lack direct causal relationships.
- ▌ Few Tokens Matter Vlm Attacks · adu2021Demonstrate that adversarial attacks on vision-language models need not target all tokens equally. Entropy-guided attacks identify high-entropy tokens (critical decision points) where perturbations have maximum impact, achieving comparable attack success with 80% fewer tokens targeted.
- ▌ Flare Fast Low Rank Attention · adu2021Implement low-rank attention routing using encode-decode factorization to achieve linear-time complexity on long sequences while maintaining compatibility with optimized attention kernels.
- ▌ Fourier Approximated Kv Cache · adu2021Training-free framework compressing KV caches using Fourier basis functions, exploiting heterogeneous transformer head roles for memory-efficient long-context LLMs.
- ▌ Fuselip Multimodal Embeddings · adu2021Build unified multimodal embeddings with a single transformer encoder processing image and text tokens together, improving performance on structure-aware tasks through early fusion.
- ▌ Grape Group Position Encoding · adu2021Unify positional encoding methods via group action theory, encompassing RoPE and ALiBi as special cases. GRAPE enables exploration of cross-subspace feature coupling—ideal when you need principled positional encoding beyond standard implementations.
- ▌ Gui 360 Desktop Agent Dataset · adu2021Enable training and evaluation of desktop computer-using agents through 1.2M action steps across diverse Windows applications, covering GUI grounding, screen parsing, and action prediction with hybrid GUI+API action space reflecting modern agent designs.
- ▌ Hybrid Reinforcement Learning · adu2021Combine sparse verifier rewards with dense reward model scores using stratified normalization to overcome limitations of either approach alone.
- ▌ Image Super Resolution Agents · adu2021Upscale any degraded image to 4K using an agentic framework that analyzes image quality, selects appropriate restoration tools, and iteratively improves results through reasoning and reflection.
- ▌ Imagine Then Plan World Model · adu2021Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models. From arXiv:2601.08955
- ▌ Knapsack Rl Budget Allocation · adu2021Improve RL training for LLMs by dynamically allocating exploration budget (rollout count) to tasks based on their difficulty and current learning status. Solves the knapsack problem of maximizing gradient signal within fixed compute budget, increasing non-zero policy gradients by 20-40% and achieving 2-4 point performance gains.
- ▌ Knowledge Agents Rl Synthesis · adu2021Build knowledge agents that generalize across grounded reasoning tasks by combining agentic data synthesis via vector search with off-policy reinforcement learning. Use multi-task training on synthetic question-answer pairs to achieve robust cross-domain performance.
- ▌ Latent Entropy Aware Decoding · adu2021Reduce hallucinations in multimodal reasoning by detecting high-entropy (uncertain) states and switching to continuous latent embeddings instead of discrete tokens. Use prior-guided visual anchoring during uncertain phases to maintain grounding.
- ▌ Latte Flow Unified Multimodal · adu2021Unify image understanding and generation with layerwise timestep experts and residual attention reuse, achieving 6x faster inference than comparable unified models while maintaining competitive performance.
- ▌ Less Is More Sparse Attention · adu2021Improves inference efficiency through training-free sparse attention using global token selection patterns aggregated from local attention heads for unified cross-head ranking.
- ▌ Let It Calm Annealed Decoding · adu2021Use dynamic temperature scheduling that explores early (high temperature) and exploits late (low temperature) during generation, improving sample efficiency in RL with verifiable rewards.
- ▌ Mars Rl Multi Agent Reasoning · adu2021Train multi-agent reasoning systems with decoupled reward signals and pipeline parallelism—enable specialized Solver/Verifier/Corrector agents to iteratively refine solutions without waiting for full trajectories, handling extended reasoning up to 320K tokens.
- ▌ Memfof Efficient Optical Flow · adu2021Reduces optical flow GPU memory 3.9× while maintaining state-of-the-art accuracy through correlation volume downsampling and dimension compensation. Enables native FullHD training with 2.09GB inference memory. Use for motion estimation in memory-constrained environments or high-resolution video processing.
- ▌ Memgovern Code Agent Learning · adu2021MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences. From arXiv:2601.06789
- ▌ Mix Reasoning Switching Modes · adu2021Improve reasoning efficiency by dynamically adjusting elaboration depth within a single response. Difficult steps receive extensive chain-of-thought; straightforward steps use concise inference. Mix reasoning modes to achieve 57-79% token savings under compute budgets while maintaining accuracy on math, STEM, and coding benchmarks.
- ▌ Monet Latent Visual Reasoning · adu2021Enable multimodal LLMs to generate and reason with latent visual embeddings as intermediate thoughts: implement supervised fine-tuning to produce continuous visual representations, then optimize via VLPO to treat embeddings as learnable actions in reinforcement learning.
- ▌ Msign Stable Rank Restoration · adu2021Prevent unrecoverable gradient explosions in LLM training by periodically restoring weight matrix stable rank through SVD-based matrix sign operations, eliminating sudden training failures without computational burden.
- ▌ Presentation Video Generation · adu2021Transform lengthy documents into fully narrated presentation videos with synchronized audio-visual delivery. Automatically segments content, generates visuals, synthesizes speech, and composes final video.