Aider LLM Leaderboards
Aider excels with LLMs skilled at writing and editing code,
and uses benchmarks to
evaluate an LLM's ability to follow instructions and edit code successfully without
human intervention.
Aider's polyglot benchmark tests LLMs on 225 challenging Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust.
files = [
'aider/website/docs/leaderboards/index.md',
'aider/website/_data/polyglot_leaderboard.yml',
]
def get_last_modified_date(file):
result = subprocess.run(['git', 'log', '-1', '--format=%ct', file], capture_output=True, text=True)
if result.returncode == 0:
timestamp = int(result.stdout.strip())
return datetime.datetime.fromtimestamp(timestamp)
return datetime.datetime.min
mod_dates = [get_last_modified_date(file) for file in files]
latest_mod_date = max(mod_dates)
cog.out(f"{latest_mod_date.strftime('%B %d, %Y.')}")
]]]-->
November 20, 2025.
1---2name: 097-index-2e12c12a3description: Quantitative benchmarks of LLM code editing skill.4---567# Aider LLM Leaderboards89Aider excels with LLMs skilled at writing and *editing* code,10and uses benchmarks to11evaluate an LLM's ability to follow instructions and edit code successfully without12human intervention.13[Aider's polyglot benchmark](https://aider.chat/2024/12/21/polyglot.html#the-polyglot-benchmark) tests LLMs on 225 challenging Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust.1415<h2 id="leaderboard-title">Aider polyglot coding leaderboard</h2>1617<div id="controls-container" style="display: flex; align-items: center; width: 100%; max-width: 800px; margin: 10px auto; gap: 10px; box-sizing: border-box; padding: 0 5px; position: relative;">18 <input type="text" id="editSearchInput" placeholder="Search..." style="flex-grow: 1; padding: 8px; border: 1px solid #ddd; border-radius: 4px;">19 <div id="view-mode-toggle" style="display: inline-flex; border: 1px solid #ccc; border-radius: 4px;">20 <button id="mode-view-btn" class="mode-button active" data-mode="view" style="padding: 8px 8px; border: none; border-radius: 3px 0 0 3px; cursor: pointer; font-size: 14px; line-height: 1.5; min-width: 50px;">View</button>21 <button id="mode-select-btn" class="mode-button" data-mode="select" style="padding: 8px 8px; border: none; background-color: #f8f9fa; border-radius: 0; cursor: pointer; border-left: 1px solid #ccc; font-size: 14px; line-height: 1.5; min-width: 50px;">Select</button>22 <button id="mode-detail-btn" class="mode-button" data-mode="detail" style="padding: 8px 8px; border: none; background-color: #f8f9fa; border-radius: 0 3px 3px 0; cursor: pointer; border-left: 1px solid #ccc; font-size: 14px; line-height: 1.5; min-width: 50px;">Detail</button>23 </div>24<button id="close-controls-btn" style="width: 18px; height: 18px; padding: 0; border: 1px solid #ddd; border-radius: 50%; background-color: transparent; cursor: pointer; display: flex; align-items: center; justify-content: center; font-size: 12px; margin-left: 4px; color: #999;">×</button>25</div>2627<table style="width: 100%; max-width: 800px; margin: auto; border-collapse: collapse; box-shadow: 0 2px 4px rgba(0,0,0,0.1); font-size: 14px;">28 <thead style="background-color: #f2f2f2;">29 <tr>30 <th style="padding: 8px; width: 40px; text-align: center; vertical-align: middle;">31 <input type="checkbox" id="select-all-checkbox" style="display: none; cursor: pointer; vertical-align: middle;">32 </th> <!-- Header checkbox added here -->33 <th style="padding: 8px; text-align: left;">Model</th>34 <th style="padding: 8px; text-align: center; width: 25%">Percent correct</th>35 <th style="padding: 8px; text-align: center; width: 25%">Cost</th>36 <th style="padding: 8px; text-align: left;" class="col-command">Command</th>37 <th style="padding: 8px; text-align: center; width: 10%" class="col-conform">Correct edit format</th>38 <th style="padding: 8px; text-align: left; width: 10%" class="col-edit-format">Edit Format</th>39 </tr>40 </thead>41 <tbody>42 {% assign max_cost = 0 %}43 {% for row in site.data.polyglot_leaderboard %}44 {% if row.total_cost > max_cost %}45 {% assign max_cost = row.total_cost %}46 {% endif %}47 {% endfor %}48 {% if max_cost == 0 %}{% assign max_cost = 1 %}{% endif %}49 {% assign edit_sorted = site.data.polyglot_leaderboard | sort: 'pass_rate_2' | reverse %}50 {% for row in edit_sorted %} {% comment %} Add loop index for unique IDs {% endcomment %}51 {% assign row_index = forloop.index0 %}52 <tr id="main-row-{{ row_index }}">53 <td style="padding: 8px; text-align: center; vertical-align: middle;">54 <button class="toggle-details" data-target="details-{{ row_index }}" style="background: none; border: none; cursor: pointer; font-size: 16px; padding: 0; vertical-align: middle;">▶</button>55 <input type="checkbox" class="row-selector" data-row-index="{{ row_index }}" style="display: none; cursor: pointer; vertical-align: middle;">56 </td>57 <td style="padding: 8px;"><span>{{ row.model }}</span></td>58 <td class="bar-cell">59 <div class="bar-viz" style="width: {{ row.pass_rate_2 }}%; background-color: rgba(40, 167, 69, 0.3); border-right: 1px solid rgba(40, 167, 69, 0.5);"></div>60 <span>{{ row.pass_rate_2 }}%</span>61 </td>62 <td class="bar-cell cost-bar-cell">63 {% if row.total_cost > 0 %}64 <div class="bar-viz cost-bar" data-cost="{{ row.total_cost }}" data-max-cost="{{ max_cost }}" style="width: 0%; background-color: rgba(13, 110, 253, 0.3); border-right: 1px solid rgba(13, 110, 253, 0.5);"></div>65 {% endif %}66 {% assign rounded_cost = row.total_cost | times: 1.0 | round: 2 %}67 <span>{% if row.total_cost == 0 or rounded_cost == 0.00 %}{% else %}${{ rounded_cost }}{% endif %}</span>68 </td>69 <td style="padding: 8px;" class="col-command"><span><code>{{ row.command }}</code></span></td>70 <td style="padding: 8px; text-align: center;" class="col-conform"><span>{{ row.percent_cases_well_formed }}%</span></td>71 <td style="padding: 8px;" class="col-edit-format"><span>{{ row.edit_format }}</span></td>72 </tr>73 <tr class="details-row" id="details-{{ row_index }}" style="display: none; background-color: #f9f9f9;">74 <td colspan="7" style="padding: 15px; border-bottom: 1px solid #ddd;">75 <ul style="margin: 0; padding-left: 20px; list-style: none; border-bottom: 1px solid #ddd;">76 {% for pair in row %}77 {% if pair[1] != "" and pair[1] != nil %}78 <li><strong>79 {% if pair[0] == 'percent_cases_well_formed' %}80 Percent cases well formed81 {% else %}82 {{ pair[0] | replace: '_', ' ' | capitalize }}83 {% endif %}84 :</strong>85 {% if pair[0] == 'command' %}<code>{{ pair[1] }}</code>{% else %}{{ pair[1] }}{% endif %}86 </li>87 {% endif %}88 {% endfor %}89 </ul>90 </td>91 </tr>92 {% endfor %}93 </tbody>94</table>9596<style>97 #leaderboard-title {98 margin-bottom: 20px; /* Add space below the title */99 }100 tr.selected {101 color: #0056b3;102 }103 table {104 table-layout: fixed;105 }106 thead {107 border-top: 1px solid #ddd; /* Add top border to header */108 }109 td, th {110 border: none; /* Remove internal cell borders */111 word-wrap: break-word;112 overflow-wrap: break-word;113 vertical-align: middle; /* Ensure consistent vertical alignment */114 }115 tbody tr {116 height: 50px; /* Set a minimum height for all data rows */117 }118 td.col-command { /* Command column */119 font-size: 12px; /* Keep font size adjustment for command column if desired, or remove */120 }121122 /* Hide new columns first on smaller screens */123 @media screen and (max-width: 991px) {124 th.col-conform, td.col-conform,125 th.col-edit-format, td.col-edit-format {126 display: none;127 }128 /* Increase width of Percent correct and Cost columns when others are hidden */129 th:nth-child(3), td:nth-child(3), /* Percent correct */130 th:nth-child(4), td:nth-child(4) { /* Cost */131 width: 33% !important; /* Override inline style */132 }133 }134135 /* Hide command column on even smaller screens */136 @media screen and (max-width: 767px) {137 th.col-command, td.col-command { /* Command column */138 display: none;139 }140 }141142 /* --- Control Styles --- */143 #controls-container {144 margin-bottom: 20px; /* Add some space below controls */145 }146147 #editSearchInput, #view-mode-select {148 padding: 8px 12px; /* Consistent padding */149 border: 1px solid #ccc; /* Slightly softer border */150 border-radius: 4px;151 font-size: 14px; /* Match table font size */152 height: 38px; /* Match height */153 box-sizing: border-box; /* Include padding/border in height */154 }155156157 .bar-cell {158 position: relative; /* Positioning context for the bar */159 padding: 8px;160 /* text-align: center; Removed */161 overflow: hidden; /* Prevent bar from overflowing cell boundaries if needed */162 }163 .cost-bar-cell {164 background-image: none; /* Remove default gradient for cost cells */165 }166 .percent-tick, .cost-tick {167 position: absolute;168 top: 50%;169 transform: translateY(10px);170 height: 8px; /* Short tick */171 width: 1px;172 background-color: rgba(170, 170, 170, 0.5); 173 z-index: 2; /* Above the bar but below the text */174 }175 .bar-viz {176 position: absolute;177 left: 0;178 top: 50%; /* Position at the middle of the cell */179 transform: translateY(-50%); /* Center the bar vertically */180 z-index: 1; /* Above background, below ticks and text */181 height: 36px;182 border-radius: 0 2px 2px 0; /* Slightly rounded end corners */183 /* Width and colors are set inline via style attribute */184 }185 /* Add a tooltip class for showing cost information on hover */186 .cost-bar-cell:hover .bar-viz[style*="background-image"] {187 animation: stripe-animation 2s linear infinite;188 }189 @keyframes stripe-animation {190 0% { background-position: 0 0; }191 100% { background-position: 20px 0; }192 }193 .bar-cell span {194 position: absolute; /* Position relative to the cell */195 left: 5px; /* Position slightly inside the left edge */196 top: 50%; /* Center vertically */197 transform: translateY(-50%); /* Adjust vertical centering */198 z-index: 3; /* Ensure text is above everything else */199 background-color: rgba(255, 255, 255, 0.7); /* Semi-transparent white background */200 padding: 0 4px; /* Add padding around the text */201 border-radius: 3px; /* Rounded corners for the text background */202 font-size: 14px; /* Adjust font size for the numbers */203 }204 .toggle-details {205 color: #888; /* Make toggle symbol more subtle */206 transition: color 0.2s; /* Smooth transition on hover */207 }208209210 /* Style for selected rows */211 tr.row-selected > td {212 background-color: #e7f3ff; /* Example light blue highlight */213 }214215 /* Ensure checkbox is vertically aligned if needed */216 .row-selector {217 vertical-align: middle;218 }219220 /* Hide rows not matching the filter */221 tr.hidden-by-mode {222 display: none !important; /* Use important to override other display styles if necessary */223 }224 tr.hidden-by-search {225 display: none !important;226 }227228 /* --- Mode Toggle Button Styles --- */229 #view-mode-toggle {230 height: 38px; /* Match input height */231 box-sizing: border-box;232 flex-shrink: 0; /* Prevent toggle from shrinking on small screens */233 }234 .mode-button {235 transition: background-color 0.2s ease-in-out, color 0.2s ease-in-out;236 white-space: nowrap; /* Prevent text wrapping */237 }238 .mode-button:not(.active) {239 background-color: #f8f9fa; /* Light grey background */240 color: #495057; /* Dark grey text */241 }242 .mode-button:not(.active):hover {243 background-color: #e2e6ea; /* Slightly darker grey on hover */244 }245246 /* Style for highlighted rows in view mode */247 tr.view-highlighted > td {248 background-color: #fffef5; /* Very light yellow/cream */249 /* Border moved to specific cell below */250 }251 /* Apply border and adjust padding ONLY for the first *visible* cell (Model name) in view mode */252 tr.view-highlighted > td:nth-child(2) {253 border-left: 4px solid #ffc107; /* Warning yellow border */254 /* Original padding is 8px. Subtract border width. */255 padding-left: 4px;256 }257</style>258259<script>260{% include leaderboard_table.js %}261</script>262263<p class="post-date" style="margin-top: 20px;">264By Paul Gauthier,265last updated266<!--[[[cog267import subprocess268import datetime269270files = [271 'aider/website/docs/leaderboards/index.md',272 'aider/website/_data/polyglot_leaderboard.yml',273]274275def get_last_modified_date(file):276 result = subprocess.run(['git', 'log', '-1', '--format=%ct', file], capture_output=True, text=True)277 if result.returncode == 0:278 timestamp = int(result.stdout.strip())279 return datetime.datetime.fromtimestamp(timestamp)280 return datetime.datetime.min281282mod_dates = [get_last_modified_date(file) for file in files]283latest_mod_date = max(mod_dates)284cog.out(f"{latest_mod_date.strftime('%B %d, %Y.')}")285]]]-->286November 20, 2025.287<!--[[[end]]]-->288</p>
tools-only