#twitter-follow { border: 0;position: fixed; top: 240px; left:0;} #live-support { border: 0;position: fixed; top: 370px; left:0;} #knowledge-base { border: 0;position: fixed; top: 170px; left:0;}

### Trading System Analysis: Backtesting report and custom measures

Updated on 2012-05-16

Properly analyzing a trading system report is crucial before trading it. Once a trading system is backtested, you will get a detailed report that displays all the measures you need to analyze and evaluate your trading system.

Backtesting Report

The strategy report of QuantShare displays the following tabs:

Summary: Displays how some important measures evolved over time (Equity, Drawdown...)
Statistics: Shows +50 measures for the backtesting process
Details: For each trading day, you can see here portfolio positions, pending orders and portfolio stats
D.W.M.Y: Performance per day, week, month and year
Profit Distribution: Shows scaled profit and loss values
Graphs: Create custom graphs
MAE/MFE: Maximum adverse excursion distribution and maximum favorable excursion distribution. Maximum loss a trade had before it was closed and maximum profit a trade had before it was closed.
S.I.M.I: Performance per sector, industry, market and index
Exit Rules: Average performance per each exit type
Monte Carlo: Allows you to perform Monte Carlo analysis.

The "Statistics" tab of the backtesting report contains many measures that should help you analyze and decide whether to consider the trading system or not.

Here are some important ones:

Sharpe Ratio:
It is a measure of risk-adjusted performance.
Optimize a trading strategy using the Sharpe ratio
Sharpe Ratio - Part 1
Sharpe Ratio - Part 2

Sortino Ratio:
This measure was developed by Frank A. Sortino. It is the same as the Sharpe ratio, except if uses the downside deviation instead of the standard deviation.
The downside deviation is calculated by taking the standard deviation of negative asset returns. It is used in the Sortino Ratio to ignore good volatility and thus provide a risk-adjusted measure without penalizing it for upward price changes.

Profit Factor:
Calculated as the profit of profitable trades divided by the losses of losing trades, the profit factor relates the amount of profit per unit of risk.
The higher the profit factor value, the better and less risky your trading system is.

Payoff Ratio:
The higher the payoff ratio the better the system. This measure is calculated by taking the system average profit per trade then dividing it by the average loss per trade.

Performance Vs Benchmark Analysis

In the "Statistics" tab, you can also see statistics that compare the performance of your trading system with an index (Beta, Alpha, R Squared and Correlation).

To calculate these measures, you will have to specify a reference index in the trading system settings.

- Select a trading system then click on "Update"
- Select "Settings" then click on "Capital"
- Type a symbol next to "Reference Symbol". Example: ^GSPC (for the S&P 500 Index)

Create a Custom Measure

- Click on "Create a metric" (If you do not see that button, click on "+" icon to extend menu)
- Type your formula in C# or JScript then click on "Save Formula"

Example: (Metric result should be associated with "fitness" variable)

if(AnnualReturn > 10 && SharpeRatio > 1 && SortinoRatio > 1.5)
{
Fitness = 1;
}

The above measure returns "1" if the trading system report has an annual return higher than 10%, a Sharpe ratio higher than 1 and a Sortino ratio above 1.5

- After you save the metric, click on "Metric" button
- Check your item then click on "OK"
- Backtest or optimize your trading system to show this metric in the main table

Custom Measure using the Money Management Tool

The money management tool allows you to create measures for advanced analysis.

- Select a trading system then click on "Update"
- Select "Money Management" tab
- Click on "Add a new money management script"

Example: (Number of trades whose return is higher than 10%)

OnEndSimulation Event: (C#)

int nb = 0;
MMPosition[] pos = Portfolio.GetAllPositions(); // Get all positions (open and closed ones)
for(int i=0;i < pos.Length;i++)
{
if(pos[i].Performance > 10) // Check position return
{
nb = nb + 1;
}
}

How a Measure Evolve Over Time

With the money management tool, you can also create time-series metrics. These metrics will be displayed in the "Summary" tab of the trading system simulation report.

In order to create a time-series metric, you should add the metric value on each trading bar. To do this, call the "Functions.AddMetric" function in the "OnEndPeriod" event.

Example: (Number of pending buy orders)

OnEndPeriod Event: (C#)

The above script is executed on each trading bar and each time it adds the number of buy pending orders to the "PendingOrders" time-series.

How to display and analyze this time-series on a chart:

- Backtest the strategy by clicking on "Simulate"
- In "Summary" tab, right click on the chart then select "Create new pane"
- Right click on the "Equity" measure then select "Remove selected graph"
- Click on the dropdown control next to "Select a time-series to drag and drop into the chart"
- Select "PendingOrders"
- Click on "Drag" icon then drag and drop this item into the new pane

Use the left bar to Like or Tweet this post. Thank you.