How to Read an Automated Trading Strategy Backtest: Metrics, Context and Limitations

Establish what was actually tested

Before interpreting the numbers, identify the test specification.

Look for:

• The market, instrument and data source.
• The start and end dates.
• The bar type, interval and trading-hours settings.
• The strategy version and parameter values.
• Initial capital, position size and any changes in exposure.
• Whether profits were reinvested.
• Commission, spread, slippage and any applicable holding costs.
• The historical order-fill model and data resolution.
• How positions still open at the end of the test were handled.

These details establish what the report describes. Without them, apparently similar reports may represent materially different experiments.

Chart setup belongs in this specification. Changing from time-based bars to Renko changes the input received by the strategy, not merely the appearance of the chart. For that distinction, see Renko vs Time-Based Charts: What Changes for an Automated Strategy?.

Also distinguish the date on which a report was generated from the last date included in its market data. A newly exported report does not necessarily contain recent trading history.

Read net profit in the context of capital, exposure and time

Net profit summarizes the financial outcome of the trades included in the report. It does not explain how that outcome was obtained.

Start by checking the reporting unit. A result expressed in account currency is different from one expressed in points, ticks or a percentage. If a percentage is shown, identify the capital base and calculation method.

Then ask:

• Over what period was the result accumulated?
• What position size produced it?
• Was size fixed or adjusted during the test?
• How much capital and margin did the setup assume?
• Were all relevant trading costs included?
• Did a small number of trades or periods account for much of the total?

The word “net” is not a substitute for checking the cost configuration. NinjaTrader documents commission templates separately from its profit statistics, so the report label alone does not establish which charges were applied.

Comparisons also need a common basis. A higher currency profit may reflect greater exposure rather than different signal quality. A longer test has more time to accumulate both gains and losses. A compounded result and a fixed-size result describe different sizing assumptions.

Treat net profit as the beginning of the reading process, not its conclusion.

Read drawdown and the equity curve together

Drawdown describes a decline from a previous peak under the report’s calculation method. Maximum drawdown identifies the largest such decline observed during the test.

The definition matters. MetaTrader 5 distinguishes balance drawdown from equity drawdown, as well as monetary and percentage measures.

A balance-based series principally reflects realized account changes. An equity-based series also reflects the changing value of open positions. A closed-trade curve can therefore leave out adverse movement that occurred before positions were closed.

Check what the chart actually plots:

• Cumulative closed-trade profit.
• Account balance.
• Account equity including open positions.
• Currency values or percentage changes.
• Results by trade number or by calendar time.

Next, read the shape and duration of the declines. How long did the series remain below a previous peak? Did it recover before the test ended? Were losses concentrated in one episode or spread across the period?

The horizontal axis is especially important. A chart indexed by trade number does not show elapsed time in the same way as a calendar-based chart. Long inactive periods can be difficult to see.

The largest historical drawdown is an observation from the tested sample. It is not a ceiling on future losses or a statement of the capital required for live trading. Nor does a visually smooth curve establish that execution assumptions were realistic.

Consider trade count and test duration as different kinds of evidence

Trade count tells you how many reported trading outcomes support the statistics. Test duration tells you how much calendar history the strategy encountered.

Neither replaces the other.

A long test can contain very few trades. A highly active system can produce many trades during a short period without encountering a broad range of conditions.

Also check what the platform counts as a trade. Orders, executions, deals and completed positions are not interchangeable. Partial entries, partial exits and reversals can affect the relationship between activity and the reported trade total. MetaTrader 5, for example, reports total trades and total deals separately.

There is no universal trade-count threshold that makes a backtest reliable. Interpretation depends on the strategy’s holding period, the variability of outcomes, the dependence between trades and the conditions represented.

Ask whether many trades were reactions to the same market event or came from overlapping positions. A large count should not automatically be read as an equally large number of independent observations.

Selective strategies should not be dismissed simply because they trade infrequently. Their limited sample should instead remain explicit in the conclusions drawn from it.

Compare the average trade with the cost assumptions

Average trade summarizes the mean outcome per reported trade. For a straightforward currency-based report, this is the total reported trade profit or loss divided by the corresponding trade count.

Check that the numerator and denominator refer to the same population and use the same cost treatment.

The average is useful, but it is not necessarily representative of a typical trade. Read it alongside:

• Average winning and losing trades.
• The proportion of winning trades.
• The largest gains and losses.
• The spread of individual outcomes.
• Position size and holding time.

A few large gains can materially influence the mean. Variable sizing can also make a currency average difficult to compare across tests.

Costs deserve their own review. Determine whether the model includes commissions, exchange or broker charges, bid-ask spread, slippage and any relevant financing or swap charges.

Avoid counting a cost twice. Spread may already be reflected in bid and ask execution prices, while another model may represent it through a separate assumption.

The practical question is how sensitive the result is to less favorable execution. If the average outcome is small relative to plausible additional costs, the reported total warrants particular scrutiny. There is no universal cost-to-average-trade ratio that settles the question for every system.

Use win rate and profit factor as supporting statistics

Win rate describes how often reported trades were profitable. It does not describe the size of the wins or losses.

Read it together with average winning trade, average losing trade and the largest losing outcomes. Frequent small gains and occasional large losses can produce a very different profile from infrequent gains and many smaller losses.

Profit factor compares gross profit with the magnitude of gross loss. It summarizes a relationship within the tested sample, but it does not explain the order of trades, the time spent in drawdown or how concentrated the gains were.

Before comparing profit factors, confirm that the reports use compatible trade definitions and cost treatment.

Neither a high win rate nor a high profit factor removes the need to examine the underlying trade list. This guide deliberately uses no universal “good” value for either statistic.

Separate long and short results

A combined report can conceal different behavior on the two sides of a strategy.

Where both directions are permitted, review the long and short components separately:

• Trade count.
• Net profit and average trade.
• Position size and time in the market.
• Distribution across the test period.
• Losing sequences and drawdown behavior where available.

A positive combined outcome does not establish that both sides contributed positively. The combined result may depend mainly on one direction.

Equal long and short counts are not a quality requirement. A strategy may intentionally trade only one direction or use different rules for each side. The question is whether the observed distribution matches its stated design.

Be careful when interpreting separate drawdowns. The largest long-side and short-side drawdowns may occur at different times, so adding them does not reproduce the combined strategy’s maximum drawdown.

Removing a weaker side after reviewing the results is also a new selection decision. It changes the system being evaluated rather than simply clarifying the original report.

Examine market regimes and the distribution of results

An aggregate result compresses the entire test into a summary. Review how outcomes were distributed across calendar periods and relevant market conditions.

Useful questions include:

• Were gains concentrated in one year or a short episode?
• How did the system behave during directional and range-bound conditions?
• Were results concentrated in higher- or lower-volatility periods?
• Did activity depend heavily on a particular session?
• Were there extended periods with little trading?
• How many trades support each subgroup?

A strategy does not need to behave identically in every environment. Its logic may depend on a specific kind of market behavior. The purpose of this review is to identify that dependence, not to demand uniform profitability.

Regime labels need definitions. If “trending” or “high volatility” is assigned only after looking at the outcome, the explanation can become hindsight-driven.

Subdividing the report also reduces the observations available in each group. Keep the trade count and period visible beside each result, and avoid treating a small subgroup as a settled conclusion.

A longer calendar history is useful context, but it does not guarantee that every relevant condition is represented.

Look for parameter stability, not just the best setting

An optimization report answers a different question from a single fixed-setting test. It shows what happened across alternative configurations.

Read beyond the highest-ranked result.

Ask whether nearby, economically sensible settings produced broadly comparable behavior or whether the selected result stands apart from its surroundings. Examine more than net profit. Trade count, drawdown, exposure and the underlying trading behavior may change as parameters change.

Parameter stability is a diagnostic, not proof of an enduring edge. Similar results across a range of settings can still come from the same favorable historical period or shared modeling assumptions.

The selection history matters too. How many configurations, filters, markets or strategy ideas were tried before the displayed result was chosen?

Research by Bailey and coauthors examines how testing and selecting among alternatives can produce misleading backtest results. The selected winner cannot be interpreted as though it were the only idea examined.

Where a report includes out-of-sample or walk-forward results, check how those periods were used. Data repeatedly consulted to revise a strategy is no longer independent of that development process. A validation label alone does not describe the full selection history.

None of these observations should be read as a claim that every Horizon Algo report includes a particular validation procedure. The published research and testing methodology describes the company’s stated process.

Keep hypothetical results separate from live trading

A backtest is a historical simulation. It is not a record of orders submitted to a live market.

Its results depend on the available data, the implementation and the assumptions used to estimate execution. NinjaTrader explicitly documents differences between historical tests and real-time operation, including data and fill-processing differences.

Price-constructed charts require particular care. For the platform-specific issues surrounding standard Renko formation and historical testing, see Renko Charts in NinjaTrader 8: What Automated Traders Need to Know.

Hypothetical results also do not reproduce the experience of financial risk. NFA’s discussion of hypothetical performance identifies hindsight, liquidity, slippage and the ability to continue following a program through losses as important limitations.

Keep the evidence categories separate:

• A backtest evaluates historical data under modeled conditions.
• Real-time simulation observes behavior without live brokerage execution.
• Live trading involves actual orders and financial exposure.

Checking that software follows its rules is not the same as establishing future profitability. Horizon Algo’s methodology likewise distinguishes implementation verification from a performance guarantee.