ta-lib doesn't calculate incrementally and may give wrong result

I use ta-lib to implement some common used indicators such as RSI and MACD in my ftap system. I implemented the indicators to recalculate all data values on update. That's to say, the indicators don't calculate incrementally, which is a feature widely supported by MetaTrader indicators. This works fine in both back testing and live trading. In back testing, as I said in my another blog, the indicators only recalculate only once for each back testing, so there is no performance penalty. And in live trading, the indicators recalculate on all data when necessary, that's fine because performance is not an issue for non-HFT trading (CPU is far much faster than net connection).

Problem comes when I port my indicators to MT4

After I wrapped ftap indicators in some MQL code to run them in MT4, I got one serious problem. When back testing in MT4 with ftap indicators, the performance is horrible. This is because, unlike back testing in ftap that all “future” data is available, MT4 only feeds “current” data to the indicators. Thus the indicators have to recalculate on every tick, so the history value is recalculated once and once upon each tick, which hits the performance badly.

I noticed that ta-lib indicator functions accept a startIdx parameter. In ftap indicators, the parameter is always 0. I tried to increase startIdx in RSI indicator, basing on current bar index. The result is completely wrong. This because standard RSI requires all past data is used. However, ta-lib doesn't remember the past value. For a 14 period RSI, a value 100 of startIdx will cause only data from 86 is used, but in standard RSI, all data from 0 to 85 are also used.

The solution

The solution is, unfortunately, stopping using ta-lib and code the algorithm by myself. But luckily, there is some MT4 indicators source code on the internet, so I just need to rewrite them in C++.

MQL indicator functions don't have such problem!

I implemented a simple indicator to use MQL function iRSI to calculate the RSI indicator. The result is correct.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],
                const double &open[],
                const double &high[],
                const double &low[],
                const double &close[],
                const long &tick_volume[],
                const long &volume[],
                const int &spread[])
{
	if(Bars<=InpRSIPeriod || InpRSIPeriod<2)
		return(0);
 
 
	for(int i = 0; i < rates_total - prev_calculated; ++i) {
		const int index = i;
		ExtRSIBuffer[index] = iRSI(NULL, 0, InpRSIPeriod, PRICE_CLOSE, index);
	}
 
	return(rates_total);
}

I don't know how MetaTrader does it. My 2 guesses,

  1. MT4 maintains a RSI on first call of iRSI
  2. MT4 recalculates on all historical data to get the value at “index”.

If any one knows how MT4 does it, let me know, thanks.

Discussion

Francesco, 2017/03/31 16:53

Hi, you did a great explanation. I have the same problem with adx indicator in ta-lib that gives me different data from the adx in the mt4. Could you help me? Have you rewrited it in C#?

Wang Qi, 2017/04/01 00:36

Hi, I didn't rewrite ADX and am still using the ta-lib version because ADX is very complicated and I don't use it very often. And, if you don't need incremental calculation, ta-lib should NOT give wrong result. The result differs with MT4 may because the data feed is different, or the start time is different.

Enter your comment. Wiki syntax is allowed:
CPAKQ