Explaining Financial ML to Main Street

It's been my experience that most people outside this field have no idea how financial machine learning works. This extends to everyday people on main street, to even sophisticated actors on Wall-Street.

To set the stage, by financial ML, I am referring to using machine learning models to forecast financial markets and profit from them.

Even many sophisticated CIOs often misunderstand the application of machine learning, it's characteristics, it's advantages and it's pit-falls.

Usually when I strike up these conversations at a cocktail party, people will think financial ML is something similar to OpenAI's ChatGPT, or some image recognition model. Most people immediately think of a single omnipresent model that knows everything. Just feed it as much data as possible and you'll become an instant millionaire. Something like HAL 9000 from 2001: A Space Odyssey, that seemingly knows everything.

This is pretty far from the truth, and the reality is extremely nuanced.

I would even go as far as to caution against the overt advertisement of machine learning toolsets to unsophisticated investors, for risk that you over promise and under-deliver.

Let me explain why this HAL 9000 world-view does not hold.

Let's take Netflix as an example. They use models to predict which movies you would most like to view based on your past habits. This is called a recommendation model. Put quite plainly, if you watch a lot of buddy cop action movies, Netflix is going to keep recommending you buddy cop action movies. Your preferences aren't going to dramatically change to rom-coms. This is a pretty simple problem.

Let's take a more complex example, ChatGPT. The human language and carrying on fluent conversations is incredibly complex, but there are underlying mechanics that govern how natural language is carried out. With the right model architecture and enough computing power - as has been recently shown - was enough to crack this problem.

There's a reason this does not extend to financial markets. The signal to noise ratio is incredibly low and will always be low. The signal in this case is alpha, or financial profits. (Or in the Netflix example, the signal is what movies you like.)

Let me tell you why. When alpha is found in the market, it's is typically arbitraged away immediately. Sophisticated actors will use tremendous leverage to stuff the alpha with as much capital such that the alpha disappears.

This is why the signal to noise ratio will always remain low. Sophisticated actors are incentivized to eat away all the alpha. This is how they make money.

So what does this mean for financial models and for financial ML in general? Well it means that you cannot simply throw a bunch of data into the model and expect to find returns. The signal to noise ratio is so low that the models will ultimately fail to find traction. It's as if you like a different type of movies every week, and by that, I mean you prefer rom-coms 50.1% of the time one week, and you prefer suspense movies 50.1% the next week.

The underlying drivers of market returns are also not stationary - they change over time. There is not one underlying function that governs the market at all times. Sometimes it's interest rates, sometimes it's debt overhang, sometimes it's oil restrictions, sometimes it could be regulatory changes which cause a structural shift in the behavior of financial markets. It's as if the English language changes from day to day - ChatGPT is an incredibly large model that has learned the English language - but it is not built to adapt to dramatic structural changes.

This leads me to how financial ML actually works in markets. The best alphas and models are specific, not general. You need to put an incredible amount of effort to crafting and finding the right signals for your models. It is only once this isolated signal is presented to this machine learning model will they really start to excel.

Think about it this way. Is there a specific time of day, a specific asset class, a specific level of volatility. Then feed this into a model to try and boost the signal.

If you just dump a bunch of data into a model, then the noise will be so high that the model will fail. Or worse, it will overfit.

So instead of having one single omni-present HAL 9000 model, you have many individual, very specific financial ML models. Each of this models is targeted a very specific source of alpha in the market.

It is also for this reason why I caution the overhype of ML models. Done right they definitely outperform more conventional models. But the reality is the hypercompetitive nature of the markets, and the resultant ever shrinking alpha is the very reason we need ML in the first place. To try and extract as much alpha from the scant signal as possible.