Decode Agent Course Reveals Three Ways to Run AI Loops
A new open-source course featuring the Decode Python agent demonstrates how optimizing the software harness rather than the model itself dramatically improves performance and cuts operational costs.
The open-source course 'Building a Coding Agent From Scratch,' published by Decoding AI, introduces a Python-based agent called Decode to demonstrate how agent architecture dictates performance. While developers often focus on model selection, LangChain's Terminal-Bench experiment proved that swapping only the surrounding software harness—while keeping the model identical—propelled a coding agent from roughly 30th place into the top five. The core agent in Decode is remarkably compact, utilizing a Pydantic AI definition of about 20 lines to compose its model, tools, and output type.
To handle different operational demands, the course outlines three distinct execution modes. The first is an interactive terminal user interface that runs in a single process and streams tokens. To prevent user input from corrupting active tool calls, Decode uses a steering queue that only injects messages at safe boundaries, specifically 'MODEL_REQUEST' or 'WOULD_STOP.' The second is a remote headless mode running on ZenML's Kitaru runtime on Google Cloud Platform, with execution handled by Modal Sandboxes. The third is an asynchronous background queue that processes tasks like Slack-triggered workflows without requiring active user supervision.
These architectural choices directly dictate the underlying economics. For instance, processing 1,000 documents of 30,000 input tokens and 500 output tokens each costs about $97 using frontier APIs priced at $3 per million input and $15 per million output tokens. Running the same workload batched on a serverless GPU at 3,000 tokens per second takes under three hours and costs roughly $13. Conversely, running Decode's default Qwen3.6 35B model on a single H200 SXM instance via Modal costs $0.001261 per second, or $4.54 per hour. Leaving this interactive setup idle overnight for ten hours while waiting for human confirmation wastes approximately $45.
For practitioners, the choice between serverless and reserved capacity hinges on demand volatility. Serverless infrastructure becomes the more economical choice when the peak-to-average demand ratio exceeds the reservation discount. While typical reservation discounts range from 2x to 5x, peak-to-average ratios for agentic development hover between 5x and 10x. Because industry surveys show reservation utilization is frequently below 30% and sometimes under 10%, serverless setups help developers avoid paying for idle peak capacity.
This is our own summary of reporting by MarkTechPost
