How are Issuers Discussing AI in their Financial Disclosures?

The surge in Artificial Intelligence (AI) and associated use cases has the potential to impact businesses that participate in our capital markets. We examined the extent to which Canadian listed issuers mention AI in their financial disclosure and the sentiment of these mentions. Through the use of Large Language Models (LLMs), our exploratory analysis suggests issuers are increasingly considering both the risks and opportunities of AI in relation to their businesses. An increasing number of issuers are mentioning AI in their Management Discussion & Analysis (MD&A) disclosure. The sentiment classification used indicates that most large issuers that mentioned AI spoke positively about its potential. However, overall sentiment has recently become more balanced as more issuers note AI’s risks.^[1]

Introduction

AI is a rapidly evolving and potentially disruptive technology that could affect many Canadian businesses. AI tools promise to help businesses improve how they operate and serve their clients and customers. But, like any new technology, AI also creates potential risks for companies that adopt it and for those that integrate it into their operations.

The opportunities and risks created by AI highlight the importance of fulsome disclosure by issuers so investors can make informed decisions. Disclosure is a core pillar of securities regulation, and the Canadian Securities Administrators (CSA) emphasized the importance of disclosure, among other key themes, in recently released guidance on how securities laws apply to AI systems in capital markets.^[2]

This note considers the extent to which large, listed issuers in Canada currently reference AI in their financial disclosures. To understand the context of these AI references, we used natural language processing (NLP) tools to analyze the sentiment of the text surrounding each AI mention. Our results suggest that many issuers are considering the rise of AI and the balance between opportunities and risks. However, our work, to date, is very much a proof of concept. We caution readers that our results, while interesting, do not assess in any way the adequacy of issuers’ disclosure with respect to AI.^[3]

Additionally, this project provided an opportunity to test new LLMs – such as those underpinning popular chatbots like OpenAI’s ChatGPT – for conducting analyses that typically use traditional NLP models. Recent research indicates that these new LLMs can perform well when carrying out NLP tasks including analyzing and summarizing text and classifying sentiment.^[4] Our results suggest LLMs can be a valuable tool for securities regulators when conducting research and analysis.

More issuers are mentioning AI in their MD&As

We observed a substantial increase in the number of S&P/TSX Composite Index issuers that mentioned AI in their MD&As over the last decade. Our analysis focused on the 225 issuers included in the S&P/TSX Composite Index as of September 2024.^[5] None of the issuers that have been operating since 2015 mentioned either “AI” or “Artificial Intelligence” in their annual MD&A filed for 2015. However, this number increased to 72 issuers (or 32% of issuers that filed) based on documents filed in 2024 (Figure 1, left panel).^[6],[7] In addition, the average number of AI mentions per document increased over the period, from one in 2016 to more than seven in 2024 (Figure 1, right panel).

Figure 1: More S&P/TSX Composite Issuers are mentioning AI in their MD&As

Notes: a) Includes all mentions of “AI” and “Artificial Intelligence”; b) Calculation of average mentions per MD&A only includes documents that mention AI or Artificial Intelligence. 
Sources: OSC staff analysis of documents sourced from www.avantisai.com. 

The combination of more issuers mentioning AI, and these issuers making more mentions per document, means that the number of times that issuers mentioned AI increased dramatically over the last decade, and particularly in documents filed in 2024 (Figure 2, left panel). Notably, issuers operating outside the Finance and Technology sectors accounted for more of this increase (discussed more below).

The increase in 2024, which mainly reflects MD&A documents filed by issuers for their 2023 financial year, coincides with rising public interest in AI. For example, Google’s search interest for “AI” in Canada was almost 14 times higher in December 2024 compared to five years earlier (Figure 2, right panel). A milestone event appears to be OpenAI’s public release of ChatGPT in November 2022.

Despite the rise in mentions, around one-third of issuers in the S&P/TSX Composite Index mentioned AI in their MD&As in the 2024 filing year. However, these were mainly the larger issuers included in the S&P/TSX Composite Index as they accounted for around one-half of the Index’s total market cap.

The observed share of issuers mentioning AI is not out of line with a broader market survey from Statistics Canada showing few Canadian businesses in 2024 were making use of AI or planning to use AI. Both results suggest that most businesses may not yet find AI relevant for their operations. However, a survey of businesses’ future intentions suggests more are planning to use AI in the future.^[8]

Figure 2: Total AI mentions increased recently coinciding with public interest in AI

Notes: a) Includes all mentions of “AI” and “Artificial Intelligence”; b) Search results for “AI” in Canada; relative search interest measured as a % of peak search interest at November 2024. 
Sources: OSC staff analysis of documents sourced from www.avantisai.com; and data from Google Trends.

Most issuers spoke positively about AI until the 2024 filing year

In addition to seeing more mentions, the 2024 filing year appears to be a turning point in the overall tone used by issuers to discuss AI. To investigate this, we conducted sentiment analysis: an NLP technique where computer models, such as LLMs, assess the tone of the text given to it. In our case, we asked OpenAI’s GPT-4o mini – one of the LLMs that powers ChatGPT – to classify the text surrounding issuers’ mentions of AI as either positive, negative or neutral. Model classification is not perfect, but GPT-4o mini performed better than the other, more traditional, sentiment analysis models that we tested: GPT-4o mini’s sentiment classification matched our researchers’ classification for around 80% of the text in our test sample (see ‘Comparison of LLMs’). Another caveat is that a model can only analyze the text provided to it, potentially missing useful context included in other parts of a filing.

According to the model, issuers’ mentions of AI in the period before the 2024 filing year were overwhelmingly positive (Figure 3). Text marked as positive generally expressed optimism about AI-driven opportunities and investments and may not have specifically related to an issuer’s own use of AI tools. The 2024 filing year, however, saw a significant increase in the number of negative AI mentions. The context of these negative mentions generally referred to risks which could range from, for example, considerations about the risk of competitors adopting AI at a faster pace, to concerns around AI’s complexity, security risks and whether tools would function as intended.

Moreover, when considering the dominant sentiment type that issuers use in their MD&As, the discussion of AI across issuers became more balanced in 2024: at most two issuers referenced AI mostly negatively in their MD&A in any filing year prior to 2024, whereas this number increased to 26 issuers in 2024 (Figure 3). In 2024, the number of issuers expressing overall negative sentiment about AI in their MD&As was equal to the number expressing positive sentiment.

One reason for higher negative sentiment around AI mentions in the 2024 filing year could be that issuers often disclose in their MD&As risk factors that are affecting their business and are potentially material for investors. The growing discussion of AI in the public narrative may have prompted more issuers to consider that the risks from AI could be material.

Figure 3: AI Sentiment was overwhelmingly positive until 2024 when more issuers mentioned risks

Sentiment has diverged between issuers in the Finance and Information sectors and those in other industries

Underpinning the shift in tone in 2024 filings was a divergence between how issuers from different sectors spoke about AI. Most of the issuers that discussed AI in their MD&A in a predominantly negative tone, and which contributed to the uptick in 2024, operate in sectors outside Finance and Information (Figure 4).^[9] By comparison, a greater share of issuers in the Finance and Information sectors discussed AI with positive and neutral sentiment in 2024 and before. One explanation for this is that businesses operating in Finance and Information appear more likely to already be using AI.^[10]

Figure 4: Recently, issuers from Information and Finance appear more positive about AI than issuers outside these sectors

Comparison of LLMs for Sentiment Analysis

The analysis underpinning this note was exploratory but highlighted the potential for LLMs to support our research and analysis, particularly when complemented by human validation.

We relied on OpenAI’s GPT-4o mini model, accessed via their API, to conduct sentiment analysis for this note. However, we first tested the model’s performance against more ‘traditional’ sentiment analysis models. These traditional models included older general purpose LLMs (e.g. BERT) and domain-specific LLMs (e.g. FinBERT), and older machine learning models (e.g. TextBlob, Vader).

In our test of 100 randomly selected three sentence text snippets from MD&As that mention either “AI” or “Artificial Intelligence”, GPT-4o mini’s sentiment classification matched that of our researchers for 83% of the sampled AI mentions. GPT-4o mini performed better overall than all other models that we tested. GPT-4o mini also showed more balanced performance than other models (e.g. GPT-3.5 Turbo) across the different sentiment types.

Table 1: Model Accuracy by Sentiment Type
Measured as the % of total AI mentions classified correctly

Our initial takeaways reiterate several best practices about using LLMs and data analysis models:

• Well-designed prompts are important for extracting informative responses from LLMs that are in a usable format. Prompts that are more specific generally yield better results.
• Clean and appropriately structured inputs to the model will lead to higher quality outputs (i.e., garbage in, garbage out). In our case, the LLM’s sentiment classification improved when text extracted from pdfs was cleaned.
• Verifying the accuracy of the model’s output before they are used is critical as outputs can often be incorrect.

Going forward, we hope to explore other uses of LLMs’ natural language processing capability, including its potential for thematic analysis and providing reasoning for its classifications.

An increasing number of issuers are discussing AI

The trends relating to AI that we observed in issuers’ MD&As appear to align with the broader trends that we see in our economy:

• More issuers are discussing AI, but further analysis is needed to understand their use of AI tools.
• The sentiment of issuers’ individual mentions of AI, while predominantly positive for many years, appears to have become more balanced as AI has gained in popularity and more issuers highlight its risks.
• Issuers from sectors with greater investment in and use of AI– such as the Finance and Information sectors – appear generally more neutral or positive about AI in their disclosures.

Our exploratory analysis highlights interesting trends about issuers’ discussion of AI in their disclosures. However, further analysis is needed to fully understand issuers’ overall position towards AI, particularly as our analysis focused on each mention of AI within issuers’ MD&A documents and not the entire text of these documents. Future analysis could also examine the reasons behind issuers’ positive and negative sentiment about AI, and the extent that their discussion about AI relates to actual investment in and use of AI tools.

AI and the OSC’s work

The opportunities and risks of AI are important considerations for securities regulators like the OSC. The disruptive nature of AI and the potential for misuse underlines the important role of regulation and governance to manage risks. The CSA’s recent guidance is intended to provide clarity to participants on how securities laws apply to AI across all areas that we regulate. AI can also help regulators achieve their mandate by offering efficiencies and enabling solutions that promote competition and help attract investment into our capital markets.

Regulatory authorities, including the OSC, are exploring how AI tools can improve regulatory work, for example, by supporting enforcement investigations, facilitating disclosure reviews, and complementing research. The exploratory analysis presented in this note is one example of an LLM use case, and it has provided promising insights about how LLMs, with appropriate guardrails and human oversight, can enhance the work of regulators.

Tongfei Li 
Research Analyst, Economic and Market Analysis
[email protected]

Senna Eswaralingam 
Senior Economist, Economic and Market Analysis
[email protected]

Kevin Yang
Manager, Research and Analytics, Economic and Market Analysis
[email protected]

Paul Redman
Chief Economist and VP, Economic and Market Analysis
[email protected]