Experience teaches us that when a breakthrough technology emerges, a flurry of new start-ups, spin-offs or company pivots follow. But given the investment needed in infrastructure, it will likely take time before we see a wave of disruptive innovations emerge outside of the incumbent tech firms. Foundational models – those with hundreds of billions of parameters – require significant ‘compute’ power and typically cost over $50 million to build and train such models[1].
Nevertheless, to borrow the metaphor popularised by Isaac Newton, “If I have seen further, it is only by standing on the shoulders of giants”, it’s likely that smaller and more nimble firms will be able to leverage the work of incumbents and build smaller and more specialised models. These smaller and less power-hungry models could be trained on first-party company or industry-specific data, with very specific applications. This would ultimately enable them to be deployed on ’edge devices’ such as smartphones. This process is called ‘knowledge distillation’ – where the knowledge from a large unwieldy model or set of models flows to a single smaller model that can be practically deployed under real-world constraints.
Indeed, platforms of distilled knowledge in the form of open-source models and datasets are already cropping up. Franco-American start-up Hugging Face offers both foundational and distilled models ranging from image classification and object detection to speech recognition and summarisation[2].
An interesting illustration of the knowledge distillation process, though paradoxically this time in reverse, is Nuance Communications. Since the late 90s, Nuance Communications has been developing and refining voice recognition systems. One of the company’s first applications, called Dragon, was a pre-curser to the virtual assistants we see in smartphones and smart-TVs today. Nuance’s automatic speech recognition and speech to text technology has been adopted in healthcare, allowing doctors to dictate patient notes, in finance, enabling fraud prevention through speech recognition, and in retail, where their messaging apps can deliver personalised customer engagements through automated chat-bots.
Microsoft completed the acquisition of Nuance Communications in July 2022 for $17.2 billion: a 23% premium to its closing price the day prior to announcement on 12 April 2021[3]. The deal has allowed Microsoft, which also owns a stake in OpenAI, to combine Nuance’s expertise with OpenAI’s GPT4 model and compute power. This has already led to some impressive innovations in the medical field, with doctors able to reduce the burden of managing their patients electronic health records thanks to the combined technologies, helping reduce the administrative paperwork and increasing the time spent on patient care[4].
[1] Source: Thematics AM, August 2023
[2] Source: https://huggingface.co/
[3] Source: https://www.bloomberg.com/news/articles/2021-04-11/microsoft-is-said-to-be-in-talks-to-buy-nuance-communications
[4] Source: https://www.nuance.com/index.html
Karen Kharmandarian and Alexandre Zilliox are portfolio managers at Thematics AM, an affiliate of Natixis Investment Managers
The provision of this material and/or reference to specific securities, sectors, or markets within this material does not constitute investment advice, or a recommendation or an offer to buy or to sell any security, or an offer of any regulated financial activity. Investors should consider the investment objectives, risks and expenses of any investment carefully before investing. The analyses, opinions, and certain of the investment themes and processes referenced herein represent the views of the portfolio manager(s) as of the date indicated. These, as well as the portfolio holdings and characteristics shown, are subject to change. There can be no assurance that developments will transpire as may be forecasted in this material. The analyses and opinions expressed by external third parties are independent and does not necessarily reflect those of Natixis Investment Managers. Past performance information presented is not indicative of future performance.
Additional notes
This material is provided by Natixis Investment Managers UK Limited (the ‘Firm’) which is authorised and regulated by the UK Financial Conduct Authority (register no. 190258). Registered Office: Natixis Investment Managers UK Limited, One Carter Lane, London, EC4V 5ER.
When permitted, the distribution of this material is intended to be made to persons as described below:
In the United Kingdom: this material is intended to be communicated to and/or directed at investment professionals and professional investors only. In Ireland: this material is intended to be communicated to and/or directed at professional investors only. In Guernsey: this material is intended to be communicated to and/or directed at only financial services providers which hold a license from the Guernsey Financial Services Commission. In Jersey: this material is intended to be communicated to and/or directed at professional investors only. In the Isle of Man: this material is intended to be communicated to and/or directed at only financial services providers which hold a license from the Isle of Man Financial Services Authority or insurers authorised under section 8 of the Insurance Act 2008.
To the extent that this material is issued by Natixis Investment Managers UK Limited, the fund, services or opinions referred to in this material are only available to the intended recipients and this material must not be relied nor acted upon by any other persons. This material is provided to the intended recipients for information purposes only. This material does not constitute an offer to the public.
It is the responsibility of each investment service provider to ensure that the offering or sale of fund shares or third party investment services to its clients complies with the relevant national law.
The above referenced entity is a business development unit of Natixis Investment Managers, the holding company of a diverse line-up of specialised investment management and distribution entities worldwide. The investment management and distribution subsidiaries of Natixis Investment Managers conduct any regulated activities only in and from the jurisdictions in which they are licensed or authorized. Their services and the products they manage are not available to all investors in all jurisdictions.
Although Natixis Investment Managers believes the information provided in this material to be reliable, including that from third party sources, it does not guarantee the accuracy, adequacy, or completeness of such information.
The provision of this material and/or reference to specific securities, sectors, or markets within this material does not constitute investment advice, or a recommendation or an offer to buy or to sell any security, or an offer of services. Investors should consider the investment objectives, risks and expenses of any investment carefully before investing. The analyses, opinions, and certain of the investment themes and processes referenced herein represent the views of the portfolio manager(s) as of the date indicated. These, as well as the portfolio holdings and characteristics shown, are subject to change. There can be no assurance that developments will transpire as may be forecasted in this material.
This material may not be distributed, published, or reproduced, in whole or in part. All amounts shown are expressed in USD unless otherwise indicated.
[1] Source: https://huggingface.co/
[2] Source: https://www.bloomberg.com/news/articles/2021-04-11/microsoft-is-said-to-be-in-talks-to-buy-nuance-communications
