The Core Problem Polar Solves<\/strong><\/h2>\nAn \u2018agent harness\u2019 is a tool like Codex CLI, Claude Code, Qwen Code, or Pi. These harnesses manage system prompts, tool formatting, context engineering, and how the agent submits patches. These details directly affect agent behavior at evaluation time.<\/p>\n
Traditional RL infrastructure requires harness logic to be rewritten behind a framework-owned environment API \u2014 typically env.init()<\/code>, env.step()<\/code>, env.reset()<\/code> in the OpenAI Gym style. Every new harness requires new integration code. That integration can also lose execution details specific to the native harness path.<\/p>\nPolar\u2019s key observation is that every LLM-based agent must call a model. That model API boundary is a common interface outside the agent itself. Instead of integrating inside the harness, Polar places a proxy at that boundary.<\/p>\n
How the Proxy Works<\/strong><\/h2>\nFor each incoming model request, the gateway proxy performs four steps:<\/strong><\/p>\n\n- Detect the provider API<\/strong> \u2014 using the request path and headers, it distinguishes Anthropic Messages, OpenAI Chat Completions, OpenAI Responses, and Google generateContent-style calls.<\/li>\n
- Normalize the request<\/strong> \u2014 converts roles, content parts, tool definitions, and generation parameters into the OpenAI Chat Completions shape used by the local inference server.<\/li>\n
- Capture token-level data<\/strong> \u2014 stores request messages, response messages, prompt token IDs, sampled response token IDs, finish reason, and log probabilities.<\/li>\n
- Return the provider shape<\/strong> \u2014 transforms the response back into the schema the harness expects.<\/li>\n<\/ol>\n
For streaming requests, Polar obtains a non-streaming upstream response and emits a synthetic provider-shaped stream. This preserves compatibility with harnesses that expect server-sent events while ensuring complete token capture.<\/p>\n
The only required change to an existing harness is pointing its model base URL at the gateway.<\/p>\n
\n![\"\"](\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/05\/Screenshot-2026-05-27-at-10.08.51-AM-1.png\")
https:\/\/arxiv.org\/pdf\/2605.24220<\/figcaption><\/figure>\n<\/div>\nArchitecture: Rollout Server and Gateway Nodes<\/strong><\/h2>\nPolar has two core components<\/strong>:<\/p>\nThe rollout server<\/strong> accepts a TaskRequest<\/code> and expands it into num_samples<\/code> independent sessions. Each session carries a session ID, task ID, timeout budget, runtime specification, agent specification, trajectory builder, evaluator, and callback URL. The server dispatches sessions to gateway nodes and accepts callbacks when sessions complete.<\/p>\nGateway nodes<\/strong> own the lifecycle of each session \u2014 starting the runtime, running the harness, building trajectories, evaluating output, and teardown. The gateway also hosts the proxy endpoint for that session\u2019s model calls, keeping completion capture tied to the session registry.<\/p>\nWithin each gateway, isolated worker pools handle INIT, RUNNING, and POSTRUN stages. A bounded READY buffer holds initialized runtimes until a run slot is available. CPU-heavy runtime preparation and evaluator prewarm proceed off the critical path, without blocking active GPU-bound agent execution. If a harness times out after model calls have been captured, the gateway still enters POSTRUN so partial traces can be recovered.<\/p>\n
Built-in evaluators include a session-completion reward, a configurable test-on-output evaluator, and a SWE-Bench\/SWE-Gym harness evaluator. Custom evaluators can be added through a registry interface.<\/p>\n
Polar currently supports Docker and rootless Apptainer runtimes. Built-in harness shortcuts include codex<\/code>, claude_code<\/code>, gemini_cli<\/code>, qwen_code<\/code>, opencode<\/code>, and pi<\/code>. <\/p>\nTrajectory Reconstruction: Per Request vs. Prefix Merging<\/strong><\/h2>\nAfter a session completes, Polar reconstructs trainable trajectories from captured model calls. <\/p>\n
Two strategies are available:<\/strong><\/p>\nThe per_request<\/code><\/strong> builder treats every model call as one independent trace. It is lossless per individual call but fragments multi-turn sessions. A single coding problem can produce hundreds of per-request traces, increasing the burden on downstream trainers.<\/p>\nThe prefix_merging<\/code><\/strong> builder reconstructs longer traces where the harness session preserves append-only conversation histories. It partitions completions into ordered chains by verifying a strict token-prefix relation between adjacent completions. Sub-agents, context compaction boundaries, and parallel agent branches naturally form separate chains. Within each merged trace, only sampled assistant tokens are marked trainable. Canonical interstitial tokens receive a loss mask of zero.<\/p>\nAblation Results<\/strong><\/h3>\nThe research team benchmarks both strategies on the same model, hardware, and topology over three training steps.<\/p>\n\n
env.init()<\/code>, env.step()<\/code>, env.reset()<\/code> in the OpenAI Gym style. Every new harness requires new integration code. That integration can also lose execution details specific to the native harness path.<\/p>\nPolar\u2019s key observation is that every LLM-based agent must call a model. That model API boundary is a common interface outside the agent itself. Instead of integrating inside the harness, Polar places a proxy at that boundary.<\/p>\n
How the Proxy Works<\/strong><\/h2>\nFor each incoming model request, the gateway proxy performs four steps:<\/strong><\/p>\n\n- Detect the provider API<\/strong> \u2014 using the request path and headers, it distinguishes Anthropic Messages, OpenAI Chat Completions, OpenAI Responses, and Google generateContent-style calls.<\/li>\n
- Normalize the request<\/strong> \u2014 converts roles, content parts, tool definitions, and generation parameters into the OpenAI Chat Completions shape used by the local inference server.<\/li>\n
- Capture token-level data<\/strong> \u2014 stores request messages, response messages, prompt token IDs, sampled response token IDs, finish reason, and log probabilities.<\/li>\n
- Return the provider shape<\/strong> \u2014 transforms the response back into the schema the harness expects.<\/li>\n<\/ol>\n
For streaming requests, Polar obtains a non-streaming upstream response and emits a synthetic provider-shaped stream. This preserves compatibility with harnesses that expect server-sent events while ensuring complete token capture.<\/p>\n
The only required change to an existing harness is pointing its model base URL at the gateway.<\/p>\n
\nhttps:\/\/arxiv.org\/pdf\/2605.24220<\/figcaption><\/figure>\n<\/div>\nArchitecture: Rollout Server and Gateway Nodes<\/strong><\/h2>\nPolar has two core components<\/strong>:<\/p>\nThe rollout server<\/strong> accepts a TaskRequest<\/code> and expands it into num_samples<\/code> independent sessions. Each session carries a session ID, task ID, timeout budget, runtime specification, agent specification, trajectory builder, evaluator, and callback URL. The server dispatches sessions to gateway nodes and accepts callbacks when sessions complete.<\/p>\nGateway nodes<\/strong> own the lifecycle of each session \u2014 starting the runtime, running the harness, building trajectories, evaluating output, and teardown. The gateway also hosts the proxy endpoint for that session\u2019s model calls, keeping completion capture tied to the session registry.<\/p>\nWithin each gateway, isolated worker pools handle INIT, RUNNING, and POSTRUN stages. A bounded READY buffer holds initialized runtimes until a run slot is available. CPU-heavy runtime preparation and evaluator prewarm proceed off the critical path, without blocking active GPU-bound agent execution. If a harness times out after model calls have been captured, the gateway still enters POSTRUN so partial traces can be recovered.<\/p>\n
Built-in evaluators include a session-completion reward, a configurable test-on-output evaluator, and a SWE-Bench\/SWE-Gym harness evaluator. Custom evaluators can be added through a registry interface.<\/p>\n
Polar currently supports Docker and rootless Apptainer runtimes. Built-in harness shortcuts include codex<\/code>, claude_code<\/code>, gemini_cli<\/code>, qwen_code<\/code>, opencode<\/code>, and pi<\/code>. <\/p>\nTrajectory Reconstruction: Per Request vs. Prefix Merging<\/strong><\/h2>\nAfter a session completes, Polar reconstructs trainable trajectories from captured model calls. <\/p>\n
Two strategies are available:<\/strong><\/p>\nThe per_request<\/code><\/strong> builder treats every model call as one independent trace. It is lossless per individual call but fragments multi-turn sessions. A single coding problem can produce hundreds of per-request traces, increasing the burden on downstream trainers.<\/p>\nThe prefix_merging<\/code><\/strong> builder reconstructs longer traces where the harness session preserves append-only conversation histories. It partitions completions into ordered chains by verifying a strict token-prefix relation between adjacent completions. Sub-agents, context compaction boundaries, and parallel agent branches naturally form separate chains. Within each merged trace, only sampled assistant tokens are marked trainable. Canonical interstitial tokens receive a loss mask of zero.<\/p>\nAblation Results<\/strong><\/h3>\nThe research team benchmarks both strategies on the same model, hardware, and topology over three training steps.<\/p>\n\n