The Benefits of Simplicity in an Age of AI Hype

Greater complexity is not always better...

In today's fast-paced world of trading, it can be tempting to chase after the most complex and sophisticated models to try and stay ahead of competition. However, greater complexity is not always better. In fact, there is a growing body of research that indicates simple is often better.

We do not need to go any further than the classic work of Kahneman, Tetlock, and Meehle, who found that "complexity may work in the odd case, but more often than not it reduces validity." I have personally noted this over the years being in teams where researchers try to outdo each other by adding more and more complex features, with the result being more complicated yet fragile models that yielded statistically insignificant changes in results.

One of the reasons why simplicity is often overlooked is that it requires hard work and education to achieve, while complexity can be sold more easily. Complex models signal effort, intellect, and innovation, whereas simple models may be dismissed as lacking in validity.

Complexity also sells better in a professional environment, where it can be used to demonstrate an individual’s intellectual prowess or a company's expertise and resourcing. In reality, however, many of the models used by hedge funds are relatively simple and the teams of PhDs hired by e.g. CTAs are often "window dressing" (with a few notable exceptions).

Models with too many parameters are often tuned to both signal and noise, reducing their statistical forecasting power.

One way to achieve simplicity in trading is to prefer parsimonious models, with a low number of parameters relative to the amount of data available. Models with too many parameters are often tuned to both signal and noise, reducing their statistical forecasting power. In trading, the temptation to include more and more parameters to fit historical data can lead to poor predictions when applied to new data generated by the same underlying process. This basically means the more parameters you specify, the less reliable your trading system and expected profits are.

The link between fundamental data and prices is also often tenuous. By focusing on price-based information, traders don't necessarily need to be subject matter experts to be successful.

Another means of achieving simplicity in trading is to focus on price-based signals. This simplifies the type of data being processed and provides faster feedback on whether signals are working or not as prices are continually updated. Other data types, such as fundamental data, generally are not. The link between fundamental data and prices is also often tenuous. By focusing on price-based information, traders don't necessarily need to be subject matter experts to be successful.

In summary, despite the appeal of complex models and strategies, the evidence supports the use of simple combinations of features. Parsimonious models and price-based signals are two ways to achieve simplicity in trading, and these approaches have the potential to be more effective and efficient than their more complex counterparts.