AIOrc: a multi-agent workflow compiler over MCP

AIOrc compiles visual workflow graphs into a single deterministic markdown document that the target model executes sequentially via Model Context Protocol. Unlike orchestrators with an active runtime (LangGraph, CrewAI, AutoGen, Claude Agent SDK), AIOrc emits the full flow in a single MCP call — eliminating inter-agent latency and preserving context coherence within a single LLM session. A second, server-verified mode (workflow.start/workflow.next) trades that single round-trip for hard enforcement: the server dispatches one step at a time and validates every transition against the graph.

Summary and positioning

Multi-agent orchestration frameworks (LangGraph, CrewAI, AutoGen, Semantic Kernel) tackle the "how to coordinate several LLMs on one task" problem from the runtime side: they keep an active control loop, call the LLM for each agent, persist shared state, and decide the next step after each response. It's flexible, but it has three costs: latency accumulated per round-trip, context fragmentation between agents, and an orchestration runtime that lives in the target.

AIOrc proposes an alternative: if the flow graph is known a priori — with its branches, cycles, and parallel forks — then it can be compiled before execution. The product of the compilation is a single markdown document that describes agents, skills, topology, and traversal rules. The target LLM loads it once and traverses the graph within its own reasoning, in a single session. There is no active runtime. No round-trips. No shared state outside the LLM's own context.

The trade-off is known and explicit: AIOrc does not support mid-run intervention, does not allow intermediate streaming between agents, and depends on the target LLM to interpret natural-language conditions and respect per-agent caps. We call this soft determinism — the flow is deterministic in structure (always the same compiled output for the same graph + input) but its traversal depends on the LLM's fidelity to the instructions.

AIOrc technical stack

AIOrc is deliberately minimalist. The underlying idea (compile before orchestrating) does not need a complex stack:

The core of the system fits in a few files:

• src/orchestrator/index.ts — Structural validation (single Start, Ends with no outgoing edges, existing agents, valid edges, correct forks), normalizer, BFS-from-Start markdown compiler.
• src/mcp/server.ts — JSON-RPC 2.0 endpoint exposing workflow and workflow.report; gated by requireProjectKey (key or public project_id).
• src/db/schema.ts — TypeScript types and full DDL: FlowNodeType = 'start' | 'agent' | 'parallel' | 'end', FlowEdge with optional condition and priority.
• src/routes/flows.ts — PUT/GET for flows per project, with whitelist and persistence in project_flows.flow_json.
• src/routes/{projects,agents,skills,users,issues,invitations,runs,auth}.ts — Multi-tenant CRUD: fork, stars, public visibility, invitations, community issues module.
• src-frontend/flow-editor/ — React flow builder: StartNode, AgentNode, ParallelNode, EndNode; edges with a condition popover; palette and skills panels.

No Redis, no queues, no workers, no containers. Single Node binary + single SQLite file + a static Vite bundle for the flow editor. The argument is: if the flow compiles in a few milliseconds and execution is done by the target LLM, we do not need orchestration infrastructure.

Why compile instead of orchestrate

Inter-agent latency

In a traditional orchestrator, the total latency of an N-agent flow is approximately:

Where each summand includes the network round-trip to the LLM provider, token generation, and the orchestrator's decision about what to do next. For 5 agents with medium-sized responses, this is usually 8-30 seconds spent in orchestration alone.

In AIOrc, the latency is:

A single local MCP call (~1-5ms) and tokenized generation of the entire flow. No pauses between steps.

Context coherence

Multi-agent frameworks typically isolate each agent's context, passing only relevant subsets via tool calls or shared state. This is deliberate — it tries to avoid context bloat and delegate cleanly — but it introduces a cost: the model never has full visibility. A downstream agent doesn't know what the upstream reasoned about, only the literal output.

AIOrc injects the entire flow as a single markdown document into the target model's context. Each step sees the previous steps and the steps ahead. This enables internal consistency, cross-references ("as task-planner said"), and cross-cutting reasoning over the entire flow.

No agent runtime at the destination

The target project does not need to install LangChain, CrewAI, or any orchestration SDK. It only needs:

1. An MCP client — Claude Code, Cursor, Continue, or any MCP-compatible IDE already has this.
2. The AIOrc stdio bridge (mcp-bridge.js, ~50 lines).
3. A rule in CLAUDE.md that tells the model to call workflow() before responding.

This reduces the blast radius: the destination remains a regular code project, not a project with an orchestration runtime woven into it.

How the compiler works

The core of the system is compileFlow(projectId, input) in src/orchestrator/index.ts. Unlike orchestrators with an active runtime, AIOrc does not execute the flow — it transcribes it into a markdown document that describes topology, agents, and rules so the target LLM executes it within its own reasoning. The graph has four node types and edges with natural-language conditions:

There are no separate primitives for "decision", "gate", or "loop". That expressiveness emerges from combining edges with conditions, priorities, and back-edges:

• Branching: multiple edges leaving the same agent, each with its condition in natural language. The LLM evaluates the agent's output and chooses a single transition.
• Gate: an agent that emits a verdict in its output (PASS/FAIL/score≥X/etc.) followed by edges with conditions that match that verdict. The structure is the same as a branch.
• Loop: an edge that points backwards (back-edge). Cycles are allowed by construction; the cap is per-agent (max_invocations, default 10, maximum 50) and the LLM enforces it by counting its own invocations.
• Fork/Join: the parallel node. Has ≥1 incoming and ≥2 outgoing edges, all taken concurrently. The LLM fires the branches in parallel (ideally one Agent tool call per branch in the same response) and waits for all of them to finish.

Compilation algorithm

1. Flow loading. Reads project_flows.flow_json and the referenced agents. If there is no flow or it is empty, returns no_flow/empty_flow error.
2. Normalization (normalizeFlow). Applies defaults: priority=1000, max_invocations clamped to [1, 50], condition normalized to a non-empty string or undefined.
3. Fail-closed validation (validateFlow). Five structural rules: exactly 1 Start (with 0 incoming, 1 outgoing), Ends with no outgoing edges, Agents with agent_id that exists in the project, edges with valid endpoints, Parallels with ≥1 incoming and ≥2 outgoing. If there are errors, returns invalid_flow with the concatenated messages — the MCP client receives them in the JSON-RPC error.
4. Skill hoisting. Collects the unique skills referenced by all agents and emits them once at the top. Each agent references them by name, avoiding token duplication.
5. Sub-agent emission. For each agent node, emits a block with name, max_invocations, description, expected_output_format, applied skills (reference to the hoisted section), and the agent's content.
6. Topology emission. BFS from Start, building the ordered list of nodes. For each non-end node, emits its outgoing edges ordered by ascending priority, showing the destination label and condition (or "no condition — fallback").
7. Execution rules. Eleven literal rules that explain to the LLM how to traverse the graph: how to choose a transition, what to do in parallel, when to terminate, how to count invocations per agent, what to do under ambiguity.
8. Run persistence. Inserts a row in runs with the complete workflow_snapshot. This gives reproducible audit: the same flow + same input generates the same output.

Cycles as back-edges, not as a primitive

Cycles are allowed by construction — there is no validation that rejects them and there is no is_loop_back in the model. The compiler does not expand or unroll anything: if the topology has a cycle code-reviewer → bug-resolver → code-reviewer, that is emitted as two edges and that's it. The target LLM sees them, evaluates the conditions on each iteration, and keeps the per-agent count.

The cap comes from the agent, not from the loop: each FlowNode of type agent has a max_invocations (default 10, clamped to 50). When the LLM reaches the maximum and the topology would invoke it again, rule 7 instructs it to cut the flow and report to the user. It is soft-determinism: enforcement lives in the instructions and depends on the model honestly counting its invocations.

Stepped mode removes that dependency. With workflow.start / workflow.next the state machine lives on the server: AIOrc dispatches one step at a time, validates every requested transition against the graph's edges, counts invocations itself and cuts the run when a cap is reached. Illegal jumps are rejected with the list of legal transitions, every dispatch is recorded as ground truth (source='executed' in telemetry), and eval cases are graded against that verified path. Compiled mode remains available when a single round-trip matters more than enforcement.

Convergence: emergent, not detected

When two branches of a fork (or a conditional branching) return to a common node, the compiler does not compute convergence points or emit the common subgraph "only once". It simply lists each edge in the topology section. The LLM sees that two transitions arrive at the same node and invokes it once after consolidating the branches. This avoids costly reachable-set analysis at the cost of delegating coordination to the target model, which already has the entire graph in its context.

Structure of the compiled workflow

The compiler's output is a markdown document with five sections, ordered so the target LLM has all the information before starting to traverse the topology. We don't use "STEP" numbering or literal verdicts — the LLM is the one that decides which transition to take according to the natural-language conditions on each edge. This is the actual skeleton generated by src/orchestrator/index.ts:

There are deliberate design decisions behind each element:

• Hoisted skills: skills are emitted once at the start, and each agent references them by name. A skill shared by 5 agents consumes tokens ×1, not ×5.
• Natural-language conditions: instead of requiring literal machine-readable verdicts (PASS/FAIL/BLOCKED), conditions are free text that the LLM evaluates against the agent's output. It is less rigid but more expressive and reduces the design cost of the flow.
• Topology as an edge list, not as a numbered tree: the LLM sees the whole graph, not a pre-linearized sequence. This allows cycles without needing LOOP primitives, and branches without needing DECISION primitives.
• Per-agent cap, not per-loop: max_invocations is per-agent and is enforced by rule 7. The LLM keeps the explicit count in its own reasoning.
• Explicit fallback: an edge without a condition is the default transition when no other matches. If all of them have conditions and none match, the workflow terminates successfully (rule 5).
• Post-output verification: each agent block with skills includes a "mandatory verification" that asks the LLM to re-read its output and validate it against the skills before continuing.

Soft determinism: why it works

AIOrc has no programmatic enforcement mechanism — there is no coordinator agent that validates each step of the target model. Flow fidelity depends entirely on the LLM respecting the compiled instructions. We call this soft determinism.

This sounds fragile, but in practice it works well for several reasons:

• LLMs follow well-structured instructions better than brittle code. An orchestrator with code that validates outputs and retries is more predictable in theory, but in practice it frequently fails on schema edge cases. A clear imperative set of instructions does not have those edge cases.
• The target model keeps the entire context. While it is executing step 4, it still sees steps 1-3 and knows what is coming in 5-7. This gives it internal consistency that an orchestrator with fragmented context does not have.
• Empirically robust patterns. "choose ONE single transition", "keep count of how many times you invoked each agent", "wait for ALL branches to finish", "if you cannot decide, stop and consult the user" — declarative instructions that modern LLMs respect with high fidelity when present in the active context.

Where soft determinism can fail (and how to mitigate it):

• An older or weaker model misinterprets a natural-language condition — Mitigation: test the flow with the target model before productizing. If Sonnet 4.5 respects it and Sonnet 3.5 does not, require a minimum version.
• The model takes two branches at once instead of choosing one — Mitigation: rule 2 ("choose ONE single transition") plus distinct priorities (priority on each edge) make the first matching condition win.
• The model exceeds an agent's cap — Mitigation: rule 7 explicitly instructs to cut + report to the user. In real tests, modern models honestly count their invocations because they have to write them in the output.
• The model misinterprets a parallel and serializes — Mitigation: rule 8 explains that it must fire all branches in the same response (via parallel Agent tool calls). If that is not technically viable, it executes them sequentially without choosing.

Comparison with 2026 frameworks

The multi-agent orchestration space changed a lot between 2024 and 2026. Claude Agent SDK (Anthropic, late 2025) and OpenAI Agents SDK (2025) appeared as official offerings from the model providers. LangGraph matured towards stateful workflows, CrewAI focused on role-based collaboration, and AutoGen (now under the AG2 community fork plus the official Microsoft Agent Framework) stays in the conversational niche.

AIOrc does not compete directly with any of them — it occupies the compile-time, single-call, MCP-native niche that no major framework currently covers:

"AIOrc works well where the graph is stable. If your workflow branches dynamically based on intermediate outputs you cannot predict, you need an orchestrator with a runtime (LangGraph, Claude Agent SDK). If it is well-defined and only varies in parameters, compiling it is faster and more auditable — and does not require installing anything in the target project beyond the MCP client you already have."

Key differences per framework

vs. Claude Agent SDK: Claude Agent SDK is Anthropic's official offering for building Claude-based agents (what the Claude Code team uses internally). It offers subagents, customizable tools, programmatic hooks, and context management. It is runtime-based and lives in the project's code. AIOrc, in contrast, defines the flow outside the target project — the destination only needs MCP. If you have authority over the destination's code and want fine programmatic control, Claude Agent SDK is more powerful. If you want to define flows centrally and reuse them across N projects without touching them, AIOrc is more practical.

vs. LangGraph: LangGraph assumes you will write Python to define the graph, and that you want state management with checkpoints, time travel, and human-in-the-loop. AIOrc discards all of that in exchange for a visual graph, persisted as JSON, executable in a single round-trip.

vs. OpenAI Agents SDK: Simpler than LangGraph, based on handoffs between agents. OpenAI only. AIOrc is model-agnostic in principle (any MCP client) but optimized for Claude.

vs. CrewAI: CrewAI shines when agents have to collaborate in a role-based way (researcher → writer → editor) with an optional manager. AIOrc does not think in terms of roles but in terms of steps of a graph.

Case study: bug vs feature with parallel QA

A realistic flow that exercises all four node types: start, several agent, a QA parallel, a review→fix cycle, and two end with different outcomes. All conditional logic lives in the edges, not in separate primitives:

How it translates to real primitives

• Bug vs feature branch: two edges leaving task-planner, each with its condition in natural language. There is no "decision" node — the LLM evaluates the planner's output and chooses one.
• Parallel QA: a parallel node with two outgoing edges to code-reviewer and security-qa. Rule 8 instructs the LLM to fire both in the same response (via concurrent Agent tool calls) and wait for the join.
• Fix cycle: an edge QA → backend-dev pointing backwards. The LLM follows it when the condition matches and respects max_invocations on backend-dev by counting its own calls.
• Outcomes: two End nodes with different outcome ("merged", "abandoned"). The LLM reports the reached outcome on termination.

Empirical results

Tested with ambiguous prompts where the intent is not explicitly declared:

• "the /api/users endpoint returns 500 when the body is empty" → planner classifies as bug → edge to bug-resolver. ✓
• "add pagination to /api/products" → feature → new-feature. ✓
• "create an endpoint that returns hello world" → feature → new-feature. ✓

The review-fix-review cycle ran 1-3 times depending on the code-reviewer's verdict, without exceeding the cap (max_invocations: 10 on backend-dev) in any of the tests. The parallel fired both QA agents concurrently in the LLM's response, and the join consolidated the verdicts before moving on.

Technical limitations

1. Context size scales linearly with the flow

Each agent embeds its full content in the compiled output. A 12-agent flow with 2KB of content each produces a ~24KB document of instructions alone (not counting the user's request or the output the model generates). For Claude Sonnet with a 200K context, this is still marginal — but flows of 30+ agents with extensive content can approach the effective limit (where the model's performance degrades due to context rot).

Mitigation: write concise agents, lean on external skills (referenced by name instead of embedded), and split large flows into hierarchical sub-flows. Hierarchical splitting is not implemented yet in AIOrc.

2. No intermediate streaming

The end user does not see partial results until the flow ends. If an intermediate agent fails, the entire output is lost. For long tasks (more than 60 seconds of generation) this is noticeable.

Mitigation: design short flows where the cost of re-running is low, and rely on the optional workflow.report for post-hoc auditing.

3. No mid-run interruption

Once the model starts executing the compiled flow, no feedback can be injected. If the user detects an error early, they have to wait until the end.

4. Parallelism delegated to the LLM

The parallel node exists in the model and the compiler emits it with the instruction to fire via concurrent Agent tool calls, but real parallelism depends on the MCP client supporting multiple tool calls per response. If the client does not, the LLM serializes the branches — it stays correct, but loses the wall-clock gain. LangGraph and CrewAI handle parallelism in orchestrator code, not in the client.

5. The per-agent cap depends on the LLM

Each agent's max_invocations is enforced by the instructions (rule 7) and depends on the LLM honestly counting its own invocations. In practice, modern LLMs respect the cap with high fidelity because they have to write the count in their own output, but there is no programmatic enforcement: if the model "lies" about how many times it invoked something, there is no external orchestrator to cut it off.

When to use AIOrc (and when not to)

Use AIOrc when

• You have bounded and predictable flows — 3 to 12 agents, with branches, cycles, and parallel forks known a priori.
• You want minimum latency per run — a single MCP call beats N runtime calls.
• The destination is already Claude (Claude Code, Cursor, or any MCP-compatible client).
• You need reproducible audit — workflow_snapshot + execution_report give full traceability.
• You want to avoid installing an orchestration runtime in the target project.

Ideal use cases: code review pipelines, on-call playbooks, deterministic ETL, QA gates with fixed criteria, documentation generation from specs, refactor pipelines.

Do NOT use AIOrc when

• You need multi-turn conversation with the user in the middle of the flow.
• You need intermediate streaming with a UI showing step-by-step progress.
• You have agents that call external tools in parallel and the results have to be dynamically merged.
• The flow is exploratory and branches dynamically based on results you cannot model a priori.
• You need human-in-the-loop with real interrupts.

For those cases: LangGraph (state-based with interrupts), CrewAI (role-based), AutoGen (conversational).

Audit logging and the EU AI Act (Article 12)

Article 12 of the EU AI Act requires high-risk AI systems to support automatic event logging across their lifetime, with logs that allow tracing how each output was produced (retention of at least six months under Articles 19/26). AIOrc's audit layer maps directly onto that requirement for agentic workflows:

• Per-step events: in verified mode every dispatch is recorded server-side — agent, transition taken (and the condition that justified it), timestamps and the caller's identity (x-user-email).
• Interpretability: the Audit page reconstructs any run as a human-readable path; the same data is available as structured JSON.
• Tamper evidence: audit exports are HMAC-SHA256 signed — any alteration after export invalidates the signature.
• Tool-call trail: calls to external MCP servers proxied through AIOrc are logged with server, tool, caller, outcome and latency.
• Retention: runs and telemetry live in the instance database with no automatic expiry; the retention policy is in your hands (the deadline for high-risk obligations moved to December 2027 under the 2026 Digital Omnibus, but the logging architecture is in place today).

This does not make a deployment "compliant" by itself — compliance is a property of the whole system — but the event log Article 12 demands is exactly what AIOrc produces as a side effect of verified execution.

Gating agent changes with evals

Evals turn "I think this change improved the agent" into a measurable verdict. The recommended working pattern:

1. Define 3-10 eval cases per flow (Usage analytics → Evals): a fixed input, the expected End outcome, and the agents that must run.
2. Before changing an agent or the flow, run the suite from any MCP client: call workflow.eval, then workflow.start with each case id. The server grades each run against the verified path — the model never grades its own work.
3. Change the agent. Run the suite again.
4. Compare pass rates in the Evals table. A drop means the change broke a path your team depends on — fix it before anyone else hits it.

Eval verdicts also appear per-run in the Audit page, so a failing case is traceable step by step.

References