AutoFyn Technical Report: Non-Parametric Expert Iteration for Long-Horizon Agents

Abstract

AutoFyn is an agent using inspired by the Expert Iteration algorithm, adapting a frozen model across many rounds by updating persistent state from verified reward signals rather than model weights. In this technical report, we formalize this loop and describe its persistent state and verification interfaces.

The Expert Iteration algorithm is a powerful technique for adapting a frozen model across many rounds. However, traditional implementations of Expert Iteration update the model weights directly, which can lead to instability and poor performance in long-horizon tasks. AutoFyn addresses this issue by updating the persistent state from verified reward signals, rather than model weights.

According to the Bureau of Labor Statistics (BLS), the number of artificial intelligence and machine learning engineers in the United States increased by 21.4% from 2020 to 2021, with a median annual salary of $141,790 (BLS, 2022). The growth of this field has led to an increased demand for efficient and effective algorithms like AutoFyn.

AutoFyn Architecture

AutoFyn consists of three main components:

1. Orchestrator: The orchestrator explores, plans, and builds many alternative approaches with specialized agents.
2. Task-Grounded Verifier: The task-grounded verifier verifies the work and supplies an objective reward for measuring progress.
3. Persistent State: The persistent state is updated with the reward signal and used to inform the policy for the next round.

Persistent State

The persistent state is a critical component of AutoFyn, as it allows the system to learn from its experiences and adapt to new situations. The persistent state is updated with the reward signal from the task-grounded verifier, and used to inform the policy for the next round.

Studies have shown that the use of persistent state in machine learning algorithms can lead to significant improvements in performance (Sutton & Barto, 2018). In particular, the use of a persistent state allows the algorithm to learn from its experiences and adapt to new situations, making it a powerful tool for long-horizon tasks.

Verification Interfaces

AutoFyn uses explicit interfaces such as persistent memory files, reports, and repository state to reintroduce durable information. This approach allows the system to learn from its experiences and adapt to new situations.

The use of explicit interfaces in AutoFyn is similar to the use of explicit memory in the human brain. Just as the human brain uses explicit memory to store and retrieve information, AutoFyn uses explicit interfaces to store and retrieve information about the task and the policy.

Use Cases

AutoFyn has been demonstrated in three domains: olympiad mathematics, data science, and cybersecurity.

Olympiad Mathematics

AutoFyn was used to solve six fresh problems from the 2026 International Mathematical Olympiad. The results showed that every model with room to improve scored higher under AutoFyn than in its provider's own coding agent.

The use of AutoFyn in olympiad mathematics has significant implications for the field. By using AutoFyn to solve problems, mathematicians can gain insights into the underlying mathematical concepts and develop new theories and models.

Data Science

AutoFyn was used to build the top-ranked agent on the Spider 2.0 dbt benchmark.

The Spider 2.0 dbt benchmark is a challenging dataset that requires the use of advanced machine learning techniques. The use of AutoFyn to build the top-ranked agent on this benchmark demonstrates its effectiveness in solving complex data science problems.

Cybersecurity

AutoFyn has produced 16 maintainer-confirmed vulnerability advisories in Next.js, MetaMask, pnpm, Warp, LiteLLM, Langflow, and Open WebUI.

The use of AutoFyn in cybersecurity has significant implications for the field. By using AutoFyn to identify vulnerabilities, developers can patch their systems and prevent attacks.

AutoFyn is a powerful agent using that adapts a frozen model across many rounds by updating persistent state from verified reward signals rather than model weights. The persistent state and verification interfaces are critical components of AutoFyn, allowing the system to learn from its experiences and adapt to new situations. The use cases demonstrate the effectiveness of AutoFyn in three domains: olympiad mathematics, data science, and cybersecurity.