Paper Trading an AI Agent: Test Before You Trust

Your agent backtested beautifully. Sharpe above 1.5, a drawdown you can stomach, a clean equity curve. None of that tells you whether the automation you actually built matches the strategy you had in your head. AI paper trading closes that gap: the agent runs against live market data, makes every decision it would make with real money, and places no orders. This guide covers what a paper run actually tests, what to watch while it runs, how long is long enough, and the criteria that separate "ready for live" from "back to the drawing board."

Why AI paper trading is a different test

For a manual trader, paper trading mainly tests the strategy and the trader's nerve. The classic pitfalls of simulated trading (treating fake money casually, overriding rules without noticing) are covered in our guide to paper trading without fooling yourself. This article is about something narrower and newer.

When an AI agent trades for you, there is a third thing on trial: the translation. You described your strategy in plain language, or assembled it from rules, and a system turned that description into monitored conditions and order logic. A backtest validates the strategy as encoded. Paper trading validates the encoding itself, on data nobody has seen yet.

Here is what that looks like in practice. You asked for "buy the dip on ETH." You meant a daily close at least 8% below the 20-day high. The agent implemented any 8% intraday drop. Both readings can produce a plausible backtest. Then a volatile week arrives and the paper log shows three entries you would never have taken manually. That is not a strategy failure. It is a translation failure, and a live-data rehearsal is the only place it surfaces before it costs money.

There is also the data itself. Backtesting an AI trading agent replays tidy historical candles. A paper run consumes the live feed, with its delays, thin weekend books, exchange hiccups and mid-candle noise. Plenty of agents behave differently on the second diet.

What to watch while the agent runs

Treat the paper period as an audit, not a waiting room. Keep a short log and check five things.

Trigger fidelity. For every signal that fired, verify the condition was genuinely met at that moment. For every quiet day, ask whether it should have fired. Missed triggers are as damning as false ones; they usually mean the condition was encoded on the wrong timeframe or the wrong price type (last trade vs. mark, close vs. wick).

Sizing. Check that each simulated order matches your sizing rule, whether that is a fixed dollar amount, 0.5–1% of equity at risk, or an ATR-based size. Watch the boundaries: minimum order sizes, lot rounding, what happens when equity changes mid-week.

Edge cases. The events backtests smooth over are the ones that hurt. Overnight gaps that open a stock through your stop level. Trading halts. A crypto move at 3am on a Sunday. If your paper window included none of these, extend it until it has at least one.

Cost realism. Compare the simulator's assumed fills against the live spread you can see on screen. If your strategy's edge is 0.4% per trade and the real spread plus fees is 0.3%, you have learned something a green equity curve concealed.

Frequency. Count trades per week against the backtest's expectation. An agent trading twice as often as its backtest is telling you the live condition is looser than the historical one.

On Obside, paper mode runs the identical agent definition on the same live data with the same risk controls; the only difference is that orders stop at the simulation boundary. Say you type: "buy $150 of ETH when RSI(14) on the 4-hour drops below 30, max two buys per week, sell half at +12%." The copilot restates the conditions, you approve them, and the agent runs as a paper agent first. A week later the log shows three RSI triggers and two executions, with the third skipped by the max-buys rule. That skipped trade is the most valuable line in the log: your constraint held under fire.

How long to run: two to four weeks, or 20 signals

The honest answer depends on your strategy's tempo, because a paper run has two jobs that finish at different speeds. Mechanics such as triggers, sizing and routing validate within a handful of events. Performance judgment needs sample size, and fewer than roughly 20 signals tells you close to nothing.

Strategy tempo	Minimum paper run	What you are really testing
Several signals per day	2 weeks	Mechanics, plus cost drag at real frequency
A few signals per week	4 weeks or 20+ signals, whichever comes later	Trigger fidelity across varied conditions
Weekly or monthly (DCA, rebalancing)	3–4 scheduled executions	Schedule, sizing, and skip-rules only

For slow strategies, do not wait a year to collect 20 events. Validate the mechanics over three or four executions and lean on the backtest for the performance verdict. For fast strategies, you may hit 20 signals in three days; run the full two weeks anyway, so the agent crosses different market conditions. Whatever the tempo, try to include one scheduled-volatility day, a CPI print or an FOMC afternoon, or for crypto a weekend session, before you call the rehearsal complete.

One rule overrides the calendar: if you change the agent's rules mid-run, the clock restarts. A paper run only certifies the version that ran it.

Graduation criteria: when to flip the switch

Go live when all of the following are true, not most of them.

1. Zero unexplained trades. Every entry and exit in the log maps to a rule you can name from memory.
2. Zero unexplained silences. Any missed trigger has a documented cause, such as exchange downtime, not a shrug.
3. Sizing correct on every fill, including the smallest and the largest.
4. At least one stress window survived — an event day, a gap, or a weekend move — with behavior you endorse.
5. Results inside the backtest's expectation band. Note what this does not say: "profitable." A mean-reversion agent can lose money through four trending weeks and be perfectly healthy. You are checking consistency with the backtest's distribution of outcomes, not the sign of four weeks of P&L.
6. You stopped wanting to intervene. If you overrode or second-guessed half the signals, the strategy is incomplete, whatever the numbers say.

Then go live deliberately small: 10–25% of your intended allocation for the first two to four live weeks, risking 0.5–1% of equity per trade. Graduation is one rung on the ladder of autonomy, not a leap to the top. On Obside that rung is literally one switch: the same agent moves from paper to live routing with its risk controls carried over unchanged, and you can cap the allocation it touches from day one.

What sends you back to the drawing board

A failed paper run is the system working. Each failure mode has a specific remedy, and none of them is "go live anyway and watch closely."

The agent traded when you wouldn't have. Translation gap. Re-word the strategy, re-approve the restated conditions, restart the paper clock. The fix belongs in the agent creation process, not in daily supervision.

You kept wanting to veto valid signals. Your gut is running a filter your rules don't contain. Extract it. "I skip longs when the market just gapped down 3%" is a codable condition; an unwritten instinct is a future 2am override.

Costs ate the edge. If simulated profits shrink to nothing at observed spreads, lower the trade frequency or retire the idea. Live trading only makes this worse.

You can't explain a decision from the log. Never put money behind behavior you cannot reconstruct. Either the logging is inadequate, which is a platform problem, or the logic is opaque to you, which is a you problem. Both are disqualifying.

Going back costs a few weeks. The live version of the same lesson costs a few weeks and a drawdown.

Where to go from here

Paper trading an AI agent is not a formality between backtest and launch; it is the only test that examines the agent you actually have rather than the strategy you think you wrote. Run it on live data, audit it like an adversary, hold the graduation line, and start small even after a clean pass. If you want the full path in one place — describe a strategy in plain language, backtest it, rehearse it in paper mode, then flip one switch to go live with risk caps intact — that workflow is what Obside was built around.

Educational content only. This is not investment advice. Trading involves risk, including possible loss of capital.