top of page
  • Julean Albidone

More Computing Power is Not What you Need

Many people think that if they just had a more powerful computer, or more GPUs, that they could build a better algorithm, and ultimately make more money in the markets.


Look at highly successful speach or image recognition algorithms (or even ChatGPT), all of which are trained on servers with hundreds or thousands of GPUs. The logic goes that you could unlock the secrets of the market if you had access to an equivalent level of hardware.


This logic is flawed. Let me tell you why.


Problems that require endless computing power are typically problems for low-noise complex-pattern data, such as vision, text or speach. Typically these patterns are so complex, that billions of parameters are required to train them to understand these complex patterns (trillions of parameters in the case of ChatGPT 4).


Financial alpha is the opposite. You are looking at extremely noisey data, and for the most part simple patterns. The trick is the patterns are unknown, closely guarded, and easily arbitraged away.


The no free lunch theorem in finance states that you need to put in a significant level of effort to be able to extract alpha. It implies that building an automated system to handle all steps of data exploration, feature engineering, ML training and tuning, is not possible or exceedingly difficult.


Focus on the basics, understanding your data, understanding who you are trading against, create thoughtful features, and test them.


A Few Caveats...


Let me state the obvious: you need computing power in proportion to your data size. For example, tick level data for the S&P 500 Futures is nearly 100GB, span that across hundreds or thousands of other securities and you are talking terabytes of data. Obviously you will need computer hardward that can handle this amount of data. But this harware just gives you a seat at the table, it doesn't by itself give you any alpha.


Conversely, if you are just dealing with end of day close data, even a moderately powerful computer can handle such an amount of data. But by the same token, this moderately powerful computer will not give you anymore alpha than the cloud infrastructure you'd need to handle tick level data.


You also need enough computing power so that you can quickly iterate on idea generation as fast as possible. The quicker the rate at which you can test ideas, the quicker you can find and deploy alpha.

Comments


bottom of page