Run-Experiment-Improve: A Guide to Testing and Optimizing Your AGI Agents

You’ve built your first autonomous agent on agi.do. You defined a complex objective, gave it a few powerful capabilities, and hit run. Watching it reason, use tools, and deliver a result feels like magic. But what happens next? How do you move from a promising proof-of-concept to a reliable, production-grade service that consistently delivers value?

The answer lies in a simple, powerful methodology: Run, Experiment, Improve.

Just like traditional software, Agentic AI isn't a "set it and forget it" technology. The most successful autonomous systems are the result of continuous refinement. The agi.do platform is designed for this iterative cycle, providing the transparency and control you need to transform a good agent into a great one.

This guide will walk you through the three phases of this loop, using our marketAnalyst agent as an example.

import { AGI } from "@do/agi";

// Initial agent definition
const marketAnalyst = new AGI({
  objective: "Analyze Q3 market trends for renewable energy and generate a report.",
  capabilities: [
    AGI.tools.webSearch,
    AGI.tools.dataAnalysis,
    AGI.tools.reportGeneration
  ]
});

const report = await marketAnalyst.run();

1. The 'Run' Phase: Establish Your Baseline

The first step is to execute your agent and gather raw, unbiased data about its performance. This isn't about judging the result yet; it's about creating a starting point for analysis.

Your Goal: To see what the agent does with its initial instructions in a real-world scenario.

How to Do It:

• Define a Clear Test Case: Start with a representative objective. For our marketAnalyst, the initial objective is concise but also quite broad.
• Execute and Observe: Run the agent and let it complete the entire autonomous workflow.
• Capture Everything: The agi.do platform provides detailed logs for every run. Pay close attention to:
  • The agent's internal monologue or "reasoning" steps.
  • Which tools it chose from its capabilities.
  • The specific parameters it used for those tools (e.g., what search queries did webSearch use?).
  • The final output.
  • Any errors or loops it encountered.

Baseline Result: Our marketAnalyst successfully runs and produces a report. However, the report covers all renewable energy types, is mostly text-based, and seems to pull from top-level news sites rather than in-depth sources. It works, but it’s not the high-quality analysis we need.

2. The 'Experiment' Phase: Analyze and Hypothesize

With baseline data in hand, you can now become a detective. Your goal is to understand why the agent behaved as it did and form hypotheses for improvement.

Your Goal: To diagnose performance gaps and propose specific changes.

How to Analyze:

1. Task Decomposition: Did the agent correctly interpret the objective? Our agent saw "renewable energy" and treated all sources equally, from solar to geothermal.
2. Tool Usage: Did it use its tools effectively? It used webSearch but didn't dig deep. It used reportGeneration but produced a basic text file, not a polished PDF.
3. Reasoning Path: Review the logs. Did it get sidetracked? Perhaps it spent too much time on a minor sub-topic because an early search result sent it down a rabbit hole.

Based on this analysis, you can form clear hypotheses.

• Hypothesis #1: "If I make the objective more specific by focusing on 'solar and wind sectors in North America,' the agent will produce a more relevant report."
• Hypothesis #2: "If I add a constraint to 'generate a 5-page summary in PDF format with charts,' the reportGeneration tool will produce a more professional output."
• Hypothesis #3: "If I give it a more specialized tool for financial data, it will find higher-quality sources than a general webSearch."

3. The 'Improve' Phase: Implement and Re-Run

This is where you turn your hypotheses into action. On the agi.do platform, you have several powerful levers to pull.

Your Goal: To modify the agent's definition and configuration based on your hypotheses.

Tweak #1: Prompt Engineering the Objective

This is your most immediate and impactful tool. An AGI agent's performance is highly sensitive to the clarity and detail of its goal.

Before:

"Analyze Q3 market trends for renewable energy and generate a report."

After:

"Conduct a detailed market analysis of Q3 trends for the solar and wind energy sectors in North America. Identify the top 3 growth drivers and 2 significant investment risks. Generate a 5-page summary report in PDF format, including at least two charts."

This improved objective provides crucial context, constraints, and format requirements.

Tweak #2: Refining Capabilities

Your agent is only as good as the tools it can use. Sometimes improvement means adding new capabilities; other times, it means taking them away to force a more optimal path.

Let's imagine you have a custom tool for accessing financial statements. You can add it to the agent's arsenal.

// Improved agent definition
const marketAnalystV2 = new AGI({
  objective: "...", // The new, detailed objective
  capabilities: [
    AGI.tools.webSearch,
    AGI.tools.dataAnalysis,
    AGI.tools.secFilings, // <-- New, specialized tool
    AGI.tools.reportGeneration
  ]
});

By providing a specialized tool like secFilings, you guide the agent toward higher-quality data sources than a generic web search alone can provide.

Run, Experiment, Improve... Again

After implementing these improvements, you go back to step one: Run the marketAnalystV2.

New Result: The agent now produces a focused, 5-page PDF report on North American solar and wind. Thanks to the secFilings tool, the analysis is backed by more robust financial data.

The process doesn't have to stop here. Perhaps the next iteration involves feeding it a list of competitor websites to analyze. Or maybe you split the task into two agents: a ResearchAgent that hands a structured dataset to a ReportWriterAgent. The possibilities are endless.

From Prototype to Production Powerhouse

Building with Autonomous General Intelligence is an iterative journey. By embracing the Run-Experiment-Improve cycle, you can methodically refine your agents from fascinating demos into intelligent, reliable, and scalable services that automate your most complex business workflows.

The agi.do platform gives you the API, the tools, and the observability you need for rapid iteration. Stop just building agents—start optimizing them.

Ready to build and refine your first autonomous agent? Get started with agi.do today.