tools-only Refactoring leaderboard
Aider's refactoring benchmark asks the LLM to refactor 89 large methods from large python classes. This is a more challenging benchmark, which tests the model's ability to output long chunks of code without skipping sections or making mistakes. It was developed to provoke and measure GPT-4 Turbo's "lazy coding" habit.
The refactoring benchmark requires a large context window to work with large source files. Therefore, results are available for fewer models.
files = [ 'aider/website/docs/leaderboards/refactor.md', 'aider/website/_data/refactor_leaderboard.yml', ]
def get_last_modified_date(file): result = subprocess.run(['git', 'log', '-1', '--format=%ct', file], capture_output=True, text=True) if result.returncode == 0: timestamp = int(result.stdout.strip()) return datetime.datetime.fromtimestamp(timestamp) return datetime.datetime.min
mod_dates = [get_last_modified_date(file) for file in files] latest_mod_date = max(mod_dates) cog.out(f"{latest_mod_date.strftime('%B %d, %Y.')}") ]]]--> April 12, 2025.