Skip to main content

What can LLMs offer the Systematic Manager?

L-L-Mentary, my dear Watson– GAM Systematic’s Chris Longworth discusses the rise of Large Language Models in the investment world, their strengths and limitations, and assesses how systematic managers can maximise the opportunities they present.

21 December 2023

Artificial intelligence (AI) models based on large language models (LLMs), such as ChatGPT and others that followed on from predecessor technology like IBM’s Watson, truly captured the popular imagination in 2023. They have also consistently made headlines and sparked wider debate – often about ethics as much as the AI itself – to an extent that few technologies have previously, certainly since the advent of the Internet. A key factor behind surging levels of interest is that everyone – from novice users to AI experts – can interact with these systems in a much more human-like fashion, compared with previous AI techniques.

Models can already play an important role

These models can already assist with many supplementary areas of the investment process. For example, they can be used to support the production of market commentary while LLM-based code completion tools can also help with programming and development work. In addition, LLMs have many direct applications in the finance sector. For example, simply asking a general-purpose language model for an investment idea will often return a suggested portfolio. This proposition may also be accompanied by a convincing investment thesis.

AI and machine learning have historically been the domain of the systematic manager. These terms encompass a wide variety of related techniques and tools, each suited for different applications. For example, ‘reinforcement learning’ attempts to determine an optimal course of action, when the likely payoff for each action may be unknown until such time as it is tried. This turns out to be very relevant to the task of choosing an optimal strategy for executing a set of trades. For other problems – such as forecasting the return of an asset over the next year – a key challenge is the lack of historical data. Here, approaches that can explicitly capture this uncertainty, such as Bayesian probabilistic models – which essentially aim to update our existing beliefs on the back of new evidence – can be effective.

Models and managers: more in common than you might imagine

It is interesting to compare the strengths and limitations of newer language-based techniques to the more traditional machine learning-based approaches commonly used by systematic managers. Perhaps surprisingly, some of the features of language-based models turn out to be more like those of discretionary managers. There are many possible explanations for this, although one intriguing possibility is that recent advances have the potential to lead to greater convergence between systematic and discretionary investment styles.

Different approaches to the same problem

Although both systematic and discretionary managers attempt to solve the same underlying investment problem, the approaches they take often vary considerably, driven by the strength and limitations of each style. For example, traditionally, the most important input into systematic portfolios has been numerical data, such as market prices and volumes. There has been recent success from incorporating a broader range of alternative data inputs, such as earnings transcripts, into systematic models. However, this is normally achieved by pre-processing the data to extract numerical features, such as sentiment indicators. These can then be directly included into the underlying numerical models. In contrast, discretionary managers often work directly with textual data, such as company reports or economic forecasts, aggregating together a variety of sources – aiming, among other things, to help counter potential biases – to form a consistent view of the world from which an investment hypothesis can then be derived.

An important advantage of numerical models is that they can incrementally react as small changes in market conditions affect the expected pay-off of any given trade. Similarly, approaches based on probabilistic models can often express some degree of confidence in the forecasts they make. This can be directly incorporated into risk management of the portfolio.

Counterintuitively, for LLMs words do come easy, numbers less so

While LLMs are very effective at integrating textual data to form a consistent view of the world, it turns out that, like humans, it can often be challenging for LLMs to process numerical data and perform accurate mathematical computation. For example, OpenAI’s GPT4 model has been shown to deliver human-level exam performance across a wide variety of subjects, including art history and wine tasting. However, when applied to the AMC, a US high school mathematics exam, it was only able to perform as well as the bottom 10% of students. One consequence of how they work is that LLMs often struggle to capture some of the benefits of current systematic approaches that are reliant on numerical computation, such as incremental reaction to new information. This is key to effective risk management but is also much harder for language-based techniques to deliver.

Balancing diversification and risk factors

Another common feature of systematic investment styles tends to be broad portfolios holding diversified positions across many markets. This is because once a model is developed, it is often relatively straightforward to reapply that same model to other assets for which similar data is available. This enables the manager to benefit from portfolio diversification without a significant increase in workload. In contrast, discretionary investment decisions are often made on the basis of a detailed study of an individual market, with the result that the process can then be harder to transfer to other securities in a different market. This generally leads to portfolios containing fewer overall positions, but with a higher degree of conviction in each. In this case, LLMs often operate more like the discretionary manager, as they can be effective at sifting through large amounts of related information to suggest a trading hypothesis that is highly specific to each market.

Show your working – tracing decisions and handling ‘what-if’ scenarios

Another important requirement for managers is the ability to explain the rationale behind trading decisions. Despite systematic investment’s 'black box' reputation, one advantage is that all trades and adjustments to positions can ultimately be traced back to the data fed into the model. Similarly, systematic models are generally well-suited to answer questions about ‘what-if’ scenarios. By feeding hypothetical future data into a model, a manager can assess how it may react across a range of different potential market environments.

Keeping it real – the need for robust reasoning from models

There has been significant research effort aimed at improving the interpretability of LLMs. For example, ‘chain-of-thought’ approaches encourage the model to provide a step-by-step breakdown of its answers. This has been shown to both improve the quality of the answer, as well as provide a mechanism for a user to understand the model’s reasoning. However, in many cases it is not clear that this breakdown actually corresponds to the internal process originally used to generate the answer, instead acting as a retrospective justification. The presence of ‘hallucinations’ in model answers – seemingly plausible statements that are in fact entirely erroneous – also remains an ongoing concern.

Despite these limitations, the rise of new language-based techniques represents a clear opportunity for the systematic manager. Systematic and discretionary investment styles have historically been highly complementary, with the broadly diversified and incremental approach of the systematic manager contrasting with the more focused and conviction-led approach of the discretionary investor. Similarly, models that have the potential to capture some of the unique advantages of the discretionary approach are likely to combine more effectively with current numerical techniques. Ultimately, this offers the potential of a unified approach that can incorporate the strengths of both for the benefit of our clients.

Important disclosures and information
The information contained herein is given for information purposes only and does not qualify as investment advice. Opinions and assessments contained herein may change and reflect the point of view of GAM in the current economic environment. No liability shall be accepted for the accuracy and completeness of the information contained herein. Past performance is no indicator of current or future trends. The mentioned financial instruments are provided for illustrative purposes only and shall not be considered as a direct offering, investment recommendation or investment advice or an invitation to invest in any GAM product or strategy. Reference to a security is not a recommendation to buy or sell that security. The securities listed were selected from the universe of securities covered by the portfolio managers to assist the reader in better understanding the themes presented. The securities included are not necessarily held by any portfolio nor represent any recommendations by the portfolio managers nor a guarantee that objectives will be realized.

This material contains forward-looking statements relating to the objectives, opportunities, and the future performance of the U.S. market generally. Forward-looking statements may be identified by the use of such words as; “believe,” “expect,” “anticipate,” “should,” “planned,” “estimated,” “potential” and other similar terms. Examples of forward-looking statements include, but are not limited to, estimates with respect to financial condition, results of operations, and success or lack of success of any particular investment strategy. All are subject to various factors, including, but not limited to general and local economic conditions, changing levels of competition within certain industries and markets, changes in interest rates, changes in legislation or regulation, and other economic, competitive, governmental, regulatory and technological factors affecting a portfolio’s operations that could cause actual results to differ materially from projected results. Such statements are forward-looking in nature and involve a number of known and unknown risks, uncertainties and other factors, and accordingly, actual results may differ materially from those reflected or contemplated in such forward-looking statements. Prospective investors are cautioned not to place undue reliance on any forward-looking statements or examples. None of GAM or any of its affiliates or principals nor any other individual or entity assumes any obligation to update any forward-looking statements as a result of new information, subsequent events or any other circumstances. All statements made herein speak only as of the date that they were made.

Certain information contained herein has been obtained from third party sources and such information has not been independently verified by GAM. No representation, warranty, or undertaking, expressed or implied, is given to the accuracy or completeness of such information by GAM or any other person. While such sources are believed to be reliable, GAM does not assume any responsibility for the accuracy or completeness of such information. GAM does not undertake any obligation to update the information contained herein as of any future date.

Dr Chris Longworth

Head of GAM Systematic
My Insights

Contact us - we'd love to hear your feedback

Active Thinking

Treasury Trove

Julian Howard

The Swiss franc – A blessing and a curse

Thomas Funk

LVMH 2023 results: What could they mean for the luxury sector?

Flavio Cereda

Investment Opinions

Contacts

Please visit our Contacts and Locations page.