{"id":530,"date":"2026-03-09T10:47:38","date_gmt":"2026-03-09T02:47:38","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=530"},"modified":"2026-03-09T10:47:38","modified_gmt":"2026-03-09T02:47:38","slug":"andrej-karpathy-open-sources-autoresearch-a-630-line-python-tool-letting-ai-agents-run-autonomous-ml-experiments-on-single-gpus","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=530","title":{"rendered":"Andrej Karpathy Open-Sources \u2018Autoresearch\u2019: A 630-Line Python Tool Letting AI Agents Run Autonomous ML Experiments on Single GPUs"},"content":{"rendered":"

Andrej Karpathy released autoresearch<\/strong>, a minimalist Python tool designed to enable AI agents to autonomously conduct machine learning experiments. The project is a stripped-down version of the nanochat<\/strong> LLM training core, condensed into a single-file repository of approximately ~630 lines of code<\/strong>. It is optimized for execution on a single NVIDIA GPU<\/strong>.<\/p>\n

The Autonomous Iteration Loop<\/strong><\/h3>\n

The framework establishes a specific division of labor between the human researcher and the AI agent. The system operates on a continuous feedback loop where progress is tracked via git commits on a feature branch.<\/p>\n

\n\n\n\n\n\n\n\n
Component<\/strong><\/td>\nResponsibility<\/strong><\/td>\nFile Format<\/strong><\/td>\n<\/tr>\n<\/thead>\n
Human<\/strong><\/td>\nIterates on high-level research instructions and constraints.<\/td>\n.md<\/code> (Markdown)<\/td>\n<\/tr>\n
AI Agent<\/strong><\/td>\nProposes and implements modifications to the training script.<\/td>\n.py<\/code> (Python)<\/td>\n<\/tr>\n
Execution<\/strong><\/td>\nConducts a fixed-length training run to evaluate the changes.<\/td>\nShell\/Python<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n

The agent reads the human-provided instructions, modifies the training code\u2014adjusting neural network architecture, optimizers, or hyperparameters\u2014and executes a training run that lasts exactly five minutes<\/strong>.<\/p>\n

Evaluation Metrics and Validation<\/strong><\/h3>\n

To ensure the agent only retains beneficial changes, the system uses bits-per-byte (BPB)<\/strong> as the primary validation metric. BPB measures the compression efficiency of the model on a validation dataset; a lower score indicates a more accurate model.<\/p>\n