Improving NinjaTrader backtest accuracy

Challenge

Recently whilst working on our Breakout System I got the stage where I wanted to prove the backtest results for accuracy.

Normally for us this involves running the strategy on a live account and backtesting the same period, we then work out the order slip (difference between what we wanted to get in at and the actual price for the fill). Using this we run the strategy live again and using the slippage run backtest; although crude if we get pretty accurate results we are happy to put the whole strategy live.

This time however the strategy works slightly differently and therefore I wanted a greater degree of accuracy. To this end I looked at deploying multiple time frames to get the next level of granularity.

Potential Solution – lower timeframe

Essentially I made my strategy the second time frame within the implementation and a more granular timeframe as the primary time frame, that way on back tests I could view the smaller timeframe for discrepancies in my strategy (in turn this should also provide more accurate fill prices).

My original strategy was on the 4hr timeframe, within the initialize() method I simply added this

Add(Instrument.FullName,PeriodType.Minute,240,1,MarketDataType.Last);

Then within the actual strategy I included two if statements (this is the OnBarUpdate method)

if (BarsInProgress == 0)

{

//Do nothing, this is the smaller timeframe

}

if(BarsInProgress == 1) // my actual timeframe for the strategy

{

//Do stuff

//Condition 1

….

EnterLong(…

….

///Condition 2

….

EnterShort(…

….

}

When running the strategy I could then use the primary timeframe to set the granularity (i.e. I could deploy everything from a tick to a minute timeframe)

I found this particularly good when utilising stoplosses as higher timeframes it often looks as if the stoploss isn’t hit when on a lower timeframe it maybe.

To take this to the next level what was really interesting is utilising the lower timeframe for the update of a Trailing Stop. As we can control the Stop on the update of each lower timeframe bar, we can in fact more accurately track a larger upward movement, combining this with a Stop aligned to a ParabloicSAR got even more interesting results and enabled us to take profit at better levels, although I caveat this with the fact that choosing the lower timeframe for the ParabolicSAR become almost an art form.

An additional option to consider to work out more accurately the difference in Bid and Ask prices instead of giving a rough spread estimate as we usually do (purely for time sake – running backtests on tick data is very time consuming), is adding the following;

Add(Instrument.FullName,PeriodType.Tick,1,MarketDataType.Bid);

Add(Instrument.FullName,PeriodType.Tick,1,MarketDataType.Ask);

Linking you long and short orders to the Bid and Ask prices will then give even more accurate results although note you will still get discrepancies as you may not get filled at the ideal market price.

Accuracy of Backtests

So what was the overall accuracy of the new lower timeframe backtests compared to the actual live account, well over a series of orders we got this to within 3 pips per order, which essentially was the difference is our less accurate spread calculation and the difference between the ideal price and our actual fill price; this was using simple 1 minute bars as the primary data period. Now if you consider our broker charges us a typical spread of 2.5 pips then this is pretty accurate.

By adding the Bid and Ask tick data we were able to get over 99% accuracy with our live account, the only challenges being some of the large orders being filled across multiple prices.

Any questions on the above don’t hesitate to get in touch; if you are also struggling to get full backtests on your systems / strategies to the tick level get in touch, we can probably run a backtest for you to the bid/ask tick tick level with several years of data available.

 

 

Posted in Development, tagged with on