# 2571 X LLM Papers 3f6d8c40

> LLM Landscape Papers (Citation ≥ 150)

- Skill: `tools-only/2571-x-llm-papers-3f6d8c40` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add tools-only/2571-x-llm-papers-3f6d8c40`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/2571-x-llm-papers-3f6d8c40/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tools-only (https://skillmd.com/u/tools-only)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/tools-only/2571-x-llm-papers-3f6d8c40

---

# LLM Landscape Papers (Citation ≥ 150)

*Generated: 2026-02-27 17:15:42*  
*Source: Semantic Scholar API — Computer Science papers only*  
*Total papers: 647*

1. [Training language models to follow instructions with human feedback📑](https://arxiv.org/abs/2203.02155): Making language models bigger does not inherently make them better at following a user's intent. [Mar 2022] (Citations: 18,579)
2. [LoRA: Low-Rank Adaptation of Large Language Models📑](https://arxiv.org/abs/2106.09685): An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. [Jun 2021] (Citations: 16,594)
3. [Chain of Thought Prompting Elicits Reasoning in Large Language Models📑](https://arxiv.org/abs/2201.11903): We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. [Jan 2022] (Citations: 15,762)
4. [Towards Deep Learning Models Resistant to Adversarial Attacks📑](https://arxiv.org/abs/1706.06083): Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples---inputs that are almost indistinguishable from natural data and yet classified incorrectly by the ... [Jun 2017] (Citations: 14,080)
5. [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks📑](https://arxiv.org/abs/2005.11401): Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. [May 2020] (Citations: 11,522)
6. [Evaluating Large Language Models Trained on Code📑](https://arxiv.org/abs/2107.03374): We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities. [Jul 2021] (Citations: 8,432)
7. [Emerging Properties in Self-Supervised Vision Transformers📑](https://arxiv.org/abs/2104.14294): In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) [16] that stand out compared to convolutional networks (convnets). [Apr 2021] (Citations: 8,254)
8. [Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning📑](https://arxiv.org/abs/2006.07733): We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. [Jun 2020] (Citations: 8,115)
9. [Visual Instruction Tuning📑](https://arxiv.org/abs/2304.08485): Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal ... [Apr 2023] (Citations: 8,007)
10. [Direct Preference Optimization: Your Language Model is Secretly a Reward Model📑](https://arxiv.org/abs/2305.18290): While large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely unsupervised ... [May 2023] (Citations: 7,325)
11. [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations📑](https://arxiv.org/abs/1909.11942): Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks. [Sep 2019] (Citations: 7,257)
12. [Scaling Laws for Neural Language Models📑](https://arxiv.org/abs/2001.08361): We study empirical scaling laws for language model performance on the cross-entropy loss. [Jan 2020] (Citations: 7,111)
13. [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models📑](https://arxiv.org/abs/2301.12597): The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models. [Jan 2023] (Citations: 7,073)
14. [Large Language Models are Zero-Shot Reasoners📑](https://arxiv.org/abs/2205.11916): Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars. [May 2022] (Citations: 6,448)
15. [ReAct: Synergizing Reasoning and Acting in Language Models📑](https://arxiv.org/abs/2210.03629): While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e. [Oct 2022] (Citations: 6,020)
16. [Self-Consistency Improves Chain of Thought Reasoning in Language Models📑](https://arxiv.org/abs/2203.11171): Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. [Mar 2022] (Citations: 5,992)
17. [Parameter-Efficient Transfer Learning for NLP📑](https://arxiv.org/abs/1902.00751): Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. [Feb 2019] (Citations: 5,937)
18. [On the Opportunities and Risks of Foundation Models📑](https://arxiv.org/abs/2108.07258): AI is undergoing a paradigm shift with the rise of models (e. [Aug 2021] (Citations: 5,921)
19. [Mamba: Linear-Time Sequence Modeling with Selective State Spaces📑](https://arxiv.org/abs/2312.00752): Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. [Dec 2023] (Citations: 5,807)
20. [DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning📑](https://arxiv.org/abs/2501.12948): General reasoning represents a long-standing and formidable challenge in artificial intelligence (AI). [Jan 2025] (Citations: 5,435)
21. [The Power of Scale for Parameter-Efficient Prompt Tuning📑](https://arxiv.org/abs/2104.08691): In this work, we explore “prompt tuning,” a simple yet effective mechanism for learning “soft prompts” to condition frozen language models to perform specific downstream tasks. [Apr 2021] (Citations: 5,144)
22. [Efficient Memory Management for Large Language Model Serving with PagedAttention📑](https://arxiv.org/abs/2309.06180): High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. [Sep 2023] (Citations: 4,726)
23. [RoFormer: Enhanced Transformer with Rotary Position Embedding📑](https://arxiv.org/abs/2104.09864): Position encoding recently has shown effective in the transformer architecture. [Apr 2021] (Citations: 4,346)
24. [Transformer-XL: Attentive Language Models beyond a Fixed-Length Context📑](https://arxiv.org/abs/1901.02860): Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling. [Jan 2019] (Citations: 4,203)
25. [A Survey of Large Language Models📑](https://arxiv.org/abs/2303.18223): Language is essentially a complex, intricate system of human expressions governed by grammatical rules. [Mar 2023] (Citations: 4,086)
26. [Sparks of Artificial General Intelligence: Early experiments with GPT-4📑](https://arxiv.org/abs/2303.12712): Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our ... [Mar 2023] (Citations: 3,960)
27. [QLoRA: Efficient Finetuning of Quantized LLMs📑](https://arxiv.org/abs/2305.14314): We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. [May 2023] (Citations: 3,948)
28. [Scaling Instruction-Finetuned Language Models📑](https://arxiv.org/abs/2210.11416): Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. [Oct 2022] (Citations: 3,919)
29. [Large language models encode clinical knowledge📑](https://arxiv.org/abs/2212.13138): Med-PaLM, a state-of-the-art large language model for medicine, is introduced and evaluated across several medical question answering tasks, demonstrating the promise of these models in this domain. [Dec 2022] (Citations: 3,655)
30. [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback📑](https://arxiv.org/abs/2204.05862): We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants. [Apr 2022] (Citations: 3,645)
31. [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness📑](https://arxiv.org/abs/2205.14135): Transformers are slow and memory-hungry on long sequences, since the time and memory complexity of self-attention are quadratic in sequence length. [May 2022] (Citations: 3,639)
32. [Learning to Prompt for Vision-Language Models📑](https://arxiv.org/abs/2109.01134): Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks. [Sep 2021] (Citations: 3,491)
33. [BEiT: BERT Pre-Training of Image Transformers📑](https://arxiv.org/abs/2106.08254): We introduce a self-supervised vision representation model BEiT, which stands for Bidirectional Encoder representation from Image Transformers. [Jun 2021] (Citations: 3,484)
34. [Tree of Thoughts: Deliberate Problem Solving with Large Language Models📑](https://arxiv.org/abs/2305.10601): Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during ... [May 2023] (Citations: 3,408)
35. [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity📑](https://arxiv.org/abs/2101.03961): In deep learning, models typically reuse the same parameters for all inputs. [Jan 2021] (Citations: 3,345)
36. [Continual Lifelong Learning with Neural Networks: A Review📑](https://arxiv.org/abs/1802.07569): Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. [Feb 2018] (Citations: 3,304)
37. [Emergent Abilities of Large Language Models📑](https://arxiv.org/abs/2206.07682): Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. [Jun 2022] (Citations: 3,253)
38. [Program Synthesis with Large Language Models📑](https://arxiv.org/abs/2108.07732): This paper explores the limits of the current generation of large language models for program synthesis in general purpose programming languages. [Aug 2021] (Citations: 3,156)
39. [Toolformer: Language Models Can Teach Themselves to Use Tools📑](https://arxiv.org/abs/2302.04761): Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. [Feb 2023] (Citations: 2,985)
40. [A Survey on Evaluation of Large Language Models📑](https://arxiv.org/abs/2307.03109): Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. [Jul 2023] (Citations: 2,964)
41. [Self-Instruct: Aligning Language Models with Self-Generated Instructions📑](https://arxiv.org/abs/2212.10560): Large “instruction-tuned” language models (i. [Dec 2022] (Citations: 2,927)
42. [Retrieval-Augmented Generation for Large Language Models: A Survey📑](https://arxiv.org/abs/2312.10997): Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. [Dec 2023] (Citations: 2,901)
43. [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers📑](https://arxiv.org/abs/2211.14730): We propose an efficient design of Transformer-based models for multivariate time series forecasting and self-supervised representation learning. [Nov 2022] (Citations: 2,880)
44. [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models📑](https://arxiv.org/abs/2304.10592): The recent GPT-4 has demonstrated extraordinary multi-modal abilities, such as directly generating websites from handwritten text and identifying humorous elements within images. [Apr 2023] (Citations: 2,845)
45. [Barlow Twins: Self-Supervised Learning via Redundancy Reduction📑](https://arxiv.org/abs/2103.03230): Self-supervised learning (SSL) is rapidly closing the gap with supervised methods on large computer vision benchmarks. [Mar 2021] (Citations: 2,825)
46. [Training Compute-Optimal Large Language Models📑](https://arxiv.org/abs/2203.15556): We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. [Mar 2022] (Citations: 2,822)
47. [Let's Verify Step by Step📑](https://arxiv.org/abs/2305.20050): In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. [May 2023] (Citations: 2,564)
48. [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation📑](https://arxiv.org/abs/2107.07651): Large-scale vision and language representation learning has shown promising improvements on various vision-language tasks. [Jul 2021] (Citations: 2,549)
49. [Universal and Transferable Adversarial Attacks on Aligned Language Models📑](https://arxiv.org/abs/2307.15043): Because"out-of-the-box"large language models are capable of generating a great deal of objectionable content, recent work has focused on aligning these models in an attempt to prevent undesirable ... [Jul 2023] (Citations: 2,525)
50. [Constitutional AI: Harmlessness from AI Feedback📑](https://arxiv.org/abs/2212.08073): As AI systems become more capable, we would like to enlist their help to supervise other AIs. [Dec 2022] (Citations: 2,501)
51. [Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks📑](https://arxiv.org/abs/2312.14238): The exponential growth of large language models (LLMs) has opened up numerous possibilities for multi-modal AGI systems. [Dec 2023] (Citations: 2,347)
52. [PaLM-E: An Embodied Multimodal Language Model📑](https://arxiv.org/abs/2303.03378): Large language models excel at a wide range of complex tasks. [Mar 2023] (Citations: 2,343)
53. [Visual Prompt Tuning📑](https://arxiv.org/abs/2203.12119): The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning. [Mar 2022] (Citations: 2,339)
54. [A survey on large language model based autonomous agents📑](https://arxiv.org/abs/2308.11432): Autonomous agents have long been a research focus in academic and industry communities. [Aug 2023] (Citations: 2,323)
55. [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions📑](https://arxiv.org/abs/2311.05232): The emergence of large language models (LLMs) has marked a significant breakthrough in natural language processing (NLP), fueling a paradigm shift in information acquisition. [Nov 2023] (Citations: 2,187)
56. [A Continual Learning Survey: Defying Forgetting in Classification Tasks📑](https://www.semanticscholar.org/paper/90e04f3ae23ca7df5f59b11453341e3db943b6f4): Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. (Citations: 2,178)
57. [Self-Supervised Learning: Generative or Contrastive📑](https://arxiv.org/abs/2006.08218): Deep supervised learning has achieved great success in the last decade. [Jun 2020] (Citations: 2,051)
58. [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone📑](https://arxiv.org/abs/2404.14219): We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of ... [Apr 2024] (Citations: 2,000)
59. [VMamba: Visual State Space Model📑](https://arxiv.org/abs/2401.10166): Designing computationally efficient network architectures remains an ongoing necessity in computer vision. [Jan 2024] (Citations: 1,749)
60. [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling📑](https://arxiv.org/abs/2304.01373): How do large language models (LLMs) develop and evolve over the course of training? [Apr 2023] (Citations: 1,709)
61. [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond📑](https://arxiv.org/abs/2308.12966): In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. [Aug 2023] (Citations: 1,701)
62. [A Survey on Contrastive Self-supervised Learning📑](https://arxiv.org/abs/2011.00362): Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. [Nov 2020] (Citations: 1,644)
63. [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models📑](https://arxiv.org/abs/2106.10199): We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified. [Jun 2021] (Citations: 1,615)
64. [OpenVLA: An Open-Source Vision-Language-Action Model📑](https://arxiv.org/abs/2406.09246): Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training ... [Jun 2024] (Citations: 1,602)
65. [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation📑](https://arxiv.org/abs/2305.01210): Program synthesis has been long studied with recent approaches focused on directly using the power of Large Language Models (LLMs) to generate code. [May 2023] (Citations: 1,512)
66. [Improving language models by retrieving from trillions of tokens📑](https://arxiv.org/abs/2112.04426): We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. [Dec 2021] (Citations: 1,503)
67. [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model📑](https://arxiv.org/abs/2401.09417): Recently the state space models (SSMs) with efficient hardware-aware designs, i. [Jan 2024] (Citations: 1,499)
68. [The Rise and Potential of Large Language Model Based Agents: A Survey📑](https://arxiv.org/abs/2309.07864): For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. [Sep 2023] (Citations: 1,472)
69. [Solving Quantitative Reasoning Problems with Language Models📑](https://arxiv.org/abs/2206.14858): Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. [Jun 2022] (Citations: 1,448)
70. [DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence📑](https://arxiv.org/abs/2401.14196): The rapid development of large language models has revolutionized code intelligence in software development. [Jan 2024] (Citations: 1,447)
71. [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters📑](https://arxiv.org/abs/2408.03314): Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. [Aug 2024] (Citations: 1,439)
72. [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day📑](https://arxiv.org/abs/2306.00890): Conversational generative AI has demonstrated remarkable promise for empowering biomedical practitioners, but current investigations focus on unimodal text. [Jun 2023] (Citations: 1,418)
73. [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time📑](https://arxiv.org/abs/2203.05482): The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set ... [Mar 2022] (Citations: 1,362)
74. [A Comprehensive Overview of Large Language Models📑](https://arxiv.org/abs/2307.06435): Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. [Jul 2023] (Citations: 1,357)
75. [Evaluating Object Hallucination in Large Vision-Language Models📑](https://arxiv.org/abs/2305.10355): Inspired by the superior language abilities of large language models (LLM), large vision-language models (LVLM) have been recently explored by integrating powerful LLMs for improving the performance ... [May 2023] (Citations: 1,350)
76. [Scaling Vision Transformers📑](https://arxiv.org/abs/2106.04560): Attention-based neural networks such as the Vision Transformer (ViT) have recently attained state-of-the-art results on many computer vision benchmarks. [Jun 2021] (Citations: 1,336)
77. [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models📑](https://arxiv.org/abs/2211.10438): Large language models (LLMs) show excellent performance but are compute- and memory-intensive. [Nov 2022] (Citations: 1,329)
78. [Improving Factuality and Reasoning in Language Models through Multiagent Debate📑](https://arxiv.org/abs/2305.14325): Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years. [May 2023] (Citations: 1,308)
79. [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models📑](https://arxiv.org/abs/2306.13394): Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image. [Jun 2023] (Citations: 1,301)
80. [Voyager: An Open-Ended Embodied Agent with Large Language Models📑](https://arxiv.org/abs/2305.16291): We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human ... [May 2023] (Citations: 1,299)
81. [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools📑](https://arxiv.org/abs/2406.12793): We introduce ChatGLM, an evolving family of large language models that we have been developing over time. [Jun 2024] (Citations: 1,290)
82. [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning📑](https://www.semanticscholar.org/paper/bf76cb3bd7a54c60138f9523ffb462a486c4807b) (Citations: 1,273)
83. [DAPO: An Open-Source LLM Reinforcement Learning System at Scale📑](https://arxiv.org/abs/2503.14476): Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning. [Mar 2025] (Citations: 1,271)
84. [Fast Inference from Transformers via Speculative Decoding📑](https://arxiv.org/abs/2211.17192): Inference from large autoregressive models like Transformers is slow - decoding K tokens takes K serial runs of the model. [Nov 2022] (Citations: 1,258)
85. [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts📑](https://arxiv.org/abs/2310.02255): Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit impressive problem-solving skills in many tasks and domains, but their ability in mathematical reasoning in visual contexts has ... [Oct 2023] (Citations: 1,254)
86. [Unifying Large Language Models and Knowledge Graphs: A Roadmap📑](https://arxiv.org/abs/2306.08302): Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and ... [Jun 2023] (Citations: 1,235)
87. [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs📑](https://arxiv.org/abs/2307.16789): Despite the advancements of open-source large language models (LLMs), e. [Jul 2023] (Citations: 1,230)
88. [Large Language Models Are Human-Level Prompt Engineers📑](https://arxiv.org/abs/2211.01910): By conditioning on natural language instructions, large language models (LLMs) have displayed impressive capabilities as general-purpose computers. [Nov 2022] (Citations: 1,228)
89. [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning📑](https://arxiv.org/abs/2205.05638): Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part ... [May 2022] (Citations: 1,226)
90. [Reproducible Scaling Laws for Contrastive Language-Image Learning📑](https://arxiv.org/abs/2212.07143): Scaling up neural networks has led to remarkable performance across a wide range of tasks. [Dec 2022] (Citations: 1,223)
91. [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling📑](https://arxiv.org/abs/2412.05271): We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements ... [Dec 2024] (Citations: 1,210)
92. [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality📑](https://arxiv.org/abs/2304.14178): Large language models (LLMs) have demonstrated impressive zero-shot abilities on a variety of open-ended tasks, while recent research has also explored the use of LLMs for multi-modal generation. [Apr 2023] (Citations: 1,191)
93. [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints📑](https://arxiv.org/abs/2305.13245): Multi-query attention (MQA), which only uses a single key-value head, drastically speeds up decoder inference. [May 2023] (Citations: 1,188)
94. [LIMA: Less Is More for Alignment📑](https://arxiv.org/abs/2305.11206): Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and reinforcement ... [May 2023] (Citations: 1,180)
95. [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality📑](https://arxiv.org/abs/2405.21060): While Transformers have been the main architecture behind deep learning's success in language modeling, state-space models (SSMs) such as Mamba have recently been shown to match or outperform ... [May 2024] (Citations: 1,179)
96. [Jailbreaking Black Box Large Language Models in Twenty Queries📑](https://arxiv.org/abs/2310.08419): There is growing interest in ensuring that large language models (LLMs) align with human values. [Oct 2023] (Citations: 1,178)
97. [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code📑](https://arxiv.org/abs/2403.07974): Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry. [Mar 2024] (Citations: 1,146)
98. [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning📑](https://arxiv.org/abs/2105.04906): Recent self-supervised methods for image representation learning are based on maximizing the agreement between embedding vectors from different views of the same image. [May 2021] (Citations: 1,145)
99. [π0: A Vision-Language-Action Flow Model for General Robot Control📑](https://arxiv.org/abs/2410.24164): Robot learning holds tremendous promise to unlock the full potential of flexible, general, and dexterous robot systems, as well as to address some of the deepest questions in artificial intelligence. [Oct 2024] (Citations: 1,138)
100. [Towards a Unified View of Parameter-Efficient Transfer Learning📑](https://arxiv.org/abs/2110.04366): Fine-tuning large pre-trained language models on downstream tasks has become the de-facto learning paradigm in NLP. [Oct 2021] (Citations: 1,126)
101. [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot📑](https://arxiv.org/abs/2301.00774): We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of ... [Jan 2023] (Citations: 1,123)
102. [From Local to Global: A Graph RAG Approach to Query-Focused Summarization📑](https://arxiv.org/abs/2404.16130): The use of retrieval-augmented generation (RAG) to retrieve relevant information from an external knowledge source enables large language models (LLMs) to answer questions over private and/or ... [Apr 2024] (Citations: 1,114)
103. [AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration📑](https://arxiv.org/abs/2306.00978): Large language models (LLMs) have transformed numerous AI applications. [Jun 2023] (Citations: 1,098)
104. [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts📑](https://arxiv.org/abs/2112.06905): Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. [Dec 2021] (Citations: 1,095)
105. [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society📑](https://arxiv.org/abs/2303.17760): The rapid advancement of chat-based language models has led to remarkable progress in complex task-solving. [Mar 2023] (Citations: 1,084)
106. [Kosmos-2: Grounding Multimodal Large Language Models to the World📑](https://arxiv.org/abs/2306.14824): We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e. [Jun 2023] (Citations: 1,084)
107. [A survey on multimodal large language models📑](https://arxiv.org/abs/2306.13549): ABSTRACT Recently, the multimodal large language model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful large language models (LLMs) as a brain to perform ... [Jun 2023] (Citations: 1,079)
108. [S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization📑](https://arxiv.org/abs/2008.07873): Recently, significant progress has been made in sequential recommendation with deep learning. [Aug 2020] (Citations: 1,042)
109. [s1: Simple test-time scaling📑](https://arxiv.org/abs/2501.19393): Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. [Jan 2025] (Citations: 1,018)
110. [A General Language Assistant as a Laboratory for Alignment📑](https://arxiv.org/abs/2112.00861): Given the broad capabilities of large language models, it should be possible to work towards a general-purpose, text-based assistant that is aligned with human values, meaning that it is helpful ... [Dec 2021] (Citations: 1,017)
111. [Towards artificial general intelligence with hybrid Tianjic chip architecture📑](https://www.semanticscholar.org/paper/56559fcd8b588357051e0d3c46631bdb435b963b) (Citations: 1,002)
112. [Parameter-efficient fine-tuning of large-scale pre-trained language models📑](https://www.semanticscholar.org/paper/76b19363b10d7ea783e4a6494eae40d73c8e9628): With the prevalence of pre-trained language models (PLMs) and the pre-training–fine-tuning paradigm, it has been continuously shown that larger models tend to yield better performance. (Citations: 986)
113. [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!📑](https://arxiv.org/abs/2310.03693): Optimizing large language models (LLMs) for downstream use cases often involves the customization of pre-trained LLMs through further fine-tuning. [Oct 2023] (Citations: 985)
114. [Teaching Large Language Models to Self-Debug📑](https://arxiv.org/abs/2304.05128): Large language models (LLMs) have achieved impressive performance on code generation. [Apr 2023] (Citations: 976)
115. [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention📑](https://arxiv.org/abs/2303.16199): We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model. [Mar 2023] (Citations: 958)
116. [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models📑](https://arxiv.org/abs/2504.10479): We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. [Apr 2025] (Citations: 951)
117. [Gorilla: Large Language Model Connected with Massive APIs📑](https://arxiv.org/abs/2305.15334): Large Language Models (LLMs) have seen an impressive wave of advances recently, with models now excelling in a variety of tasks, such as mathematical reasoning and program synthesis. [May 2023] (Citations: 948)
118. [Bias and Fairness in Large Language Models: A Survey📑](https://arxiv.org/abs/2309.00770): Abstract Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our ... [Sep 2023] (Citations: 940)
119. [Detecting hallucinations in large language models using semantic entropy📑](https://www.semanticscholar.org/paper/f82f49c20c6acc69f884f05e3a9f1ceea91061ce): Large language model (LLM) systems, such as ChatGPT1 or Gemini2, can show impressive reasoning and question-answering capabilities but often ‘hallucinate’ false outputs and unsubstantiated answers3,4. (Citations: 927)
120. [RWKV: Reinventing RNNs for the Transformer Era📑](https://arxiv.org/abs/2305.13048): Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. [May 2023] (Citations: 911)
121. [Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation📑](https://arxiv.org/abs/2211.06687): Contrastive learning has shown remarkable success in the field of multimodal representation learning. [Nov 2022] (Citations: 908)
122. [Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data📑](https://arxiv.org/abs/1703.11008): One of the defining properties of deep learning is that models are chosen to have many more parameters than available training data. [Mar 2017] (Citations: 907)
123. [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model📑](https://arxiv.org/abs/2306.03341): We introduce Inference-Time Intervention (ITI), a technique designed to enhance the"truthfulness"of large language models (LLMs). [Jun 2023] (Citations: 901)
124. [Scaling Vision with Sparse Mixture of Experts📑](https://arxiv.org/abs/2106.05974): Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated excellent scalability in Natural Language Processing. [Jun 2021] (Citations: 898)
125. [Automatic Chain of Thought Prompting in Large Language Models📑](https://arxiv.org/abs/2210.03493): Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. [Oct 2022] (Citations: 891)
126. [WizardCoder: Empowering Code Large Language Models with Evol-Instruct📑](https://arxiv.org/abs/2306.08568): Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. [Jun 2023] (Citations: 889)
127. [Simplified State Space Layers for Sequence Modeling📑](https://arxiv.org/abs/2208.04933): Models using structured state space sequence (S4) layers have achieved state-of-the-art performance on long-range sequence modeling tasks. [Aug 2022] (Citations: 887)
128. [Large Language Models Can Be Easily Distracted by Irrelevant Context📑](https://arxiv.org/abs/2302.00093): Large language models have achieved impressive performance on various natural language processing tasks. [Feb 2023] (Citations: 877)
129. [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models📑](https://arxiv.org/abs/2309.01219): While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs ... [Sep 2023] (Citations: 876)
130. [Large Language Models: A Survey📑](https://arxiv.org/abs/2402.06196): Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022. [Feb 2024] (Citations: 834)
131. [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting📑](https://arxiv.org/abs/2305.04388): Large Language Models (LLMs) can achieve strong performance on many tasks by producing step-by-step reasoning before giving a final output, often referred to as chain-of-thought reasoning (CoT). [May 2023] (Citations: 833)
132. [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension📑](https://arxiv.org/abs/2307.16125): Based on powerful Large Language Models (LLMs), recent generative Multimodal Large Language Models (MLLMs) have gained prominence as a pivotal research area, exhibiting remarkable capability for both ... [Jul 2023] (Citations: 829)
133. [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions📑](https://arxiv.org/abs/2212.10509): Prompting-based large language models (LLMs) are surprisingly powerful at generating natural language reasoning steps or Chains-of-Thoughts (CoT) for multi-step question answering (QA). [Dec 2022] (Citations: 823)
134. [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers📑](https://arxiv.org/abs/1811.04918): The fundamental learning theory behind neural networks remains largely open. [Nov 2018] (Citations: 820)
135. [Yi: Open Foundation Models by 01.AI📑](https://arxiv.org/abs/2403.04652): We introduce the Yi model family, a series of language and multimodal models that demonstrate strong multi-dimensional capabilities. [Mar 2024] (Citations: 791)
136. [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models📑](https://arxiv.org/abs/2310.01728): Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. [Oct 2023] (Citations: 790)
137. [A Comparative Study on Transformer vs RNN in Speech Applications📑](https://arxiv.org/abs/1909.06317): Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS). [Sep 2019] (Citations: 789)
138. [RegionCLIP: Region-based Language-Image Pretraining📑](https://arxiv.org/abs/2112.09106): Contrastive language-image pretraining (CLIP) using image-text pairs has achieved impressive results on image classification in both zero-shot and transfer learning set-tings. [Dec 2021] (Citations: 788)
139. [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey📑](https://arxiv.org/abs/2403.14608): Large models represent a groundbreaking advancement in multiple appl

…(truncated)