FCA publishes AI good practice guide – what regulated firms must do now

The FCA's AI Update publication explains how the existing regulatory framework sets a strong foundation for how AI adoption should continue to support market stability and good consumer outcomes.  

• Under safety, security and robustness, firms are subject to the FCA's Principles for Businesses (PRIN), the FCA's Threshold Conditions (COND), Senior Management Arrangements, Systems and Controls (SYSC), operational resilience, and the outsourcing and critical third parties regime.  
• Under fairness, appropriate transparency and explainability, firms must comply with PRIN, Consumer Duty, the Guidance for firms on the fair treatment of vulnerable customers, COND, and related ICO guidance and the Equality Act 2010.  
• Accountability and governance is governed by the Senior Managers and Certification Regime (SM&CR), and contestability and redress is governed by the Dispute Resolution: Complaints' Sourcebook (DISP).  

Firms' use of AI is also subject to legal frameworks overseen by other regulators.  No new rules are needed: the framework is already in force, and the FCA's publication makes clear it is actively applying it.

Where firms are deploying AI-supported tools, they remain responsible for consumer outcomes, including outcomes they deliver through any products, services, communications and support.  When deployed safely and responsibly, AI can deliver significant benefits to consumers by improving efficiency, reducing costs, and enabling more accessible and tailored services.  

In good practice, firms start with a clear consumer problem, target market, and intended outcome before deciding whether AI is the right tool and in what form, for example, agentic or generative.  Firms in good practice also carefully consider which tasks AI can support or automate, which should remain staff-led, and where the AI tool should abstain, redirect or escalate to a person, building proportionate technical checks and continuous monitoring into the AI system rather than only relying on a person to review, approve, challenge or override its outputs.  

The area for improvement the FCA identified was stark as some firms built AI tools without establishing the consumer problem, target market or foreseeable harms, raising concerns that the product would not meet the needs of the consumers likely to use it.

Case study A: One firm wanted to make information for consumers more consistent. Before introducing AI, it defined the outcomes it wanted, considered other approaches, and set clear success measures. It then ran a narrowly scoped pilot to assess the benefits and limitations of the use case before deciding whether to scale deployment. This helped the firm ensure AI addressed a defined need and was the appropriate tool, rather than adopting it as an objective in itself.

Firms in good practice tested for unfair or unjustified differences across relevant consumer groups and scenarios. Depending on the use case, they considered factors such as the data and variables used, and compared outcomes and error rates. They addressed any issues by rebalancing the data, adjusting thresholds or adding controls, and then retested to make sure those changes improved consumer outcomes without creating new gaps, keeping ongoing monitoring for bias.  Good practice also included testing outcomes across different consumer characteristics, including vulnerability and digital or financial capability, to support inclusive design.  

An area for improvement the FCA identified concerns guardrails: some firms focused on technical model guardrails with limited consideration of how to support good consumer outcomes.

Case study B: One firm tested whether indicators linked to vulnerability, such as accessibility requirements, influenced fraud suspicion or claim acceptance. The results helped improve the model and reduce unintended outcomes.

Case study C: One firm used guardrails to successfully reduce model drift and unexpected responses. However, the tool then presented overly generic outcomes for consumers. In this instance, the narrowly designed guardrail failed to consider what good consumer outcomes were and steered some consumers towards inappropriate options.

Smaller firms note: It may not always be proportionate or feasible to test at scale. Smaller firms need to judge what is proportionate for them. The FCA saw smaller firms use sample model outputs, add vulnerability markers to test cases, and compare outputs across scenarios, helping them spot potential bias or unfair variation without needing the same volume of testing as larger firms.  Proportionately filtered and quality-assured inputs can be useful where resources are limited, with good-quality sample data being vital when testing whether consumer-facing AI tools are producing appropriate outputs.

Firms should explain how they have used AI in their products, services and journeys in a way consumers can understand, so they can make effective and properly informed decisions.  Firms should also consider whether their records, explanations and monitoring are sufficient to support challenge and help them deal with complaints.  

Good practice includes making clear to consumers that they are using an AI-enabled tool, and explaining what it could and could not do, and when the consumer should look for help from another source.  Firms in good practice also kept records of consumer prompts, outputs, model versions, applied filters and escalation decisions, helping them monitor outcomes, handle complaints, support explainability and investigate where a consumer challenged an outcome.

Areas for improvement included: some firms assumed their AI outputs were clear and helpful without testing whether different consumer groups understood them, including consumers with lower financial capability, accessibility needs or vulnerability characteristics.  Other firms overstated the accuracy, reliability or role of AI systems, or were unclear about whether AI or a person performed a particular function, which could mislead consumers and undermine informed decision-making and trust.

Case study D: One firm used a large language model (LLM) tool to help consumers understand whether they might be accepted for credit. The firm had clear, concise explainers around the tool and in response to consumer prompts at the point of use, to clarify what the tool could and could not do. Where the tool could not act, it explained why and signposted to wider resources, such as free debt advice or other appropriate support, and offered an option to speak to a member of staff.

Case study E: One firm used an LLM tool to develop or refine letters responding to consumer queries and complaints, with case officers reviewing and sending the letters. The tool identified language and issues consumers found hard to understand and suggested clearer wording. Before deployment, the firm tested the AI-generated communications with a small sample of different consumers to assess whether the new style helped people understand the information and improved consumer outcomes against KPIs such as net promoter score and complaint resolution. Consumers said the new style was clearer and they got the information sooner, even when the firm had rejected their complaint.

To reduce risk, some firms automatically excluded consumers with vulnerability indicators from AI journeys, this may be appropriate in some scenarios, but could unfairly exclude those consumers if the firm does not provide another way for them to get support or benefits that meet their needs.

Good practice involves using AI tools to spot potential vulnerability indicators, such as key words in consumer responses, spending patterns, or signs a consumer is struggling with a digital journey.  Where appropriate, firms in good practice allowed consumers to choose between AI-enabled or non-AI support for straightforward tasks, and consumers could switch channels without having to repeat information, maintaining continuity and consumer control.  

Areas for improvement include: some firms placed too much confidence in an LLM's ability to identify text-only indicators of vulnerability, particularly temporary ones. Firms should test their systems thoroughly to make sure they can identify gaps in approaches and make changes, such as adding vulnerability indicators or questions the LLM could ask.  Some automated interfaces also made it harder for consumers to get support, with problems including unclear responses, not addressing the consumer's issue and not allowing them to move to another channel.

Case study F: One firm developed an LLM chatbot to supplement its consumer support channels. It worked with frontline staff to decide which vulnerability indicators the tool should look for, then adapted the initial questions the chatbot would ask consumers and developed a live list of proxy terms the tool flags to staff. The firm had clear plans for how to support flagged consumers, whether through the AI tool or by referring them to a human adviser, and monitored whether this improved outcomes.

Case study G: One firm's AI tool found spending patterns that indicated potential consumer vulnerability. The firm asked its ethics committee to decide whether it should continue providing credit in those circumstances and to agree a consistent approach across the business in line with its regulatory requirements. Once the firm understood the patterns and made a decision, it refined its models.

To govern effectively, firms need to have proportionate oversight, clear accountability and risk-based controls across the AI lifecycle and firms are responsible for AI systems they develop, procure and deploy, including their outputs and outcomes.  Firms should adopt proportionate governance models that cover the full AI lifecycle:

• initial scoping; design and development;
• testing and validation; deployment;
• ongoing monitoring; and decommissioning.

Good practice involves firms taking a risk-based approach to challenging plans for new deployments, involving staff from areas across the business in testing and sign-off, with formal evaluation, clear accountability and executive approval helping to identify and mitigate risks before launching new AI systems.

The FCA identified retrospective governance as a specific area for improvement as firms designed and deployed AI use cases without first considering how to govern them, then tried to fit them into rigid existing frameworks after key design decisions had already been made, thus making it harder to show who was accountable for decisions, manage AI-specific risks or tailor oversight to the use case.  Governance operating in isolation was also identified as a concern, as some firms only had governance for specific AI lifecycle stages, such as post-launch, or governed AI decisions in isolation, leaving gaps in auditability, documentation and risk management.

Case study H: One firm established an independent review panel made up of colleagues from different business areas to challenge proposed AI use cases. Panel members considered the use case from different perspectives, including human ownership and bias, and could recommend additional controls before deployment.

Case study I: One firm largely used its established model risk and third-party governance processes for AI but added targeted controls where it found gaps, including a specialist group to approve high-risk AI model changes, AI-specific questions in supplier assessments, and contractual obligations and regular checks of key suppliers. This allowed the firm to build on existing governance rather than creating a separate new framework.

Case study J: One firm deployed AI tools before it had established clear governance arrangements. When risks emerged, multiple teams thought oversight sat elsewhere and there was no clear route for people to escalate issues or challenge decisions. Retrofitting governance after deployment proved difficult and left gaps in accountability, making it harder for the firm to show how it had identified and managed AI-specific risks.

Smaller firms note: Smaller firms can scale and tailor their governance frameworks to their size, capability and risk profile rather than copying large firm models, allowing them to deploy AI safely by focusing in-house expertise where oversight and challenge are most needed, using phased and iterative AI introduction, simpler risk tiering, clear accountability and proportionate review processes, escalating only higher risks.

To govern AI effectively, firms need ongoing monitoring, clear escalation routes and trained oversight functions, so they can assess outputs, real-world outcomes and evolving risks, and intervene where needed.  

Firms in good practice adopted monitoring approaches that tracked real-world outcomes and impacts, and adapted to evolving risks such as model drift.  Examples of continuous monitoring tools the FCA has seen in good practice include:

• continuous monitoring dashboards with thresholds and benchmarks;
• dedicated post-implementation monitoring groups;
• spot checks and automated scoring frameworks to detect model drift or harmful AI outputs;
• logging and audit trails of AI interactions; and
• feedback from staff on whether they used or modified an AI output and why.

Areas for improvement:

• where firms monitored technical metrics rather than outcomes, they had less chance of meaningful oversight.
• some firms treated governance as a point-in-time exercise and did not follow up with sufficient monitoring or review after deployment.  
• some firms had reactive approaches, meaning they could only identify harm after issues arose rather than anticipating it.  
• some firms did not have enough expertise in their second and third-line oversight functions to identify or challenge emerging risks, as their training was generic or limited to awareness, without supporting effective oversight, and they did not have effective escalation triggers or intervention mechanisms.

Case study K: One firm assessed AI-generated claims information against a structured set of criteria, including completeness, faithfulness to source information, conciseness, adequacy of reasoning, and safety. It combined checks by staff with an automated assessment, including LLM-as-a-judge, to reduce the risk that characteristics or biases shared by models from the same developer could affect assessment outcomes.

Firms in good practice gave staff ongoing training and carried out capability reviews of their human-in-the-loop and oversight functions. Some training was role-specific:

• for senior stakeholders, it covered AI-related regulatory expectations, governance responsibilities, risk appetite, accountability and escalation routes; for technical teams, it covered model risk, data quality, ethics, guardrails, controls, testing and monitoring requirements;
• for human-in-the-loop roles, it covered understanding a system's purpose and limitations, how to interpret and challenge outputs, managing automation bias, judging issues independently, and knowing when to escalate or override.

Deploying AI may create new operational and governance challenges, but the FCA does not expect senior managers to be technical specialists, they should, however, take reasonable steps to understand the risks arising from AI use within their area of responsibility, obtain appropriate information and expertise where necessary, provide effective oversight and challenge, and ensure that governance, controls and escalation arrangements remain appropriate to the risks presented.  

Good practice involves firms mapping responsibilities for AI across senior management functions (SMF), establishing clear accountability, governance and oversight for AI risks. Some firms highlighted the importance of an ownership structure tailored to each AI use case, rather than assigning a single owner to each model. Roles could include technical, research and business owners, with a senior person accountable for overall oversight.  Importantly, accountability should be assigned at the beginning, with senior managers owning risks from the design stage onwards for each AI use case.

Areas for improvement:

• senior managers delegated responsibility for AI development entirely to technical teams without sufficient understanding or effective challenge. which FCA was particularly concerning for higher-risk AI use cases, including those affecting consumers.  
• some firms had no clear link between risk identification and accountability, making it harder to ensure risks were owned, escalated and addressed appropriately and limiting senior managers' ability to demonstrate that they understood the risk, challenged it appropriately and took reasonable steps to manage it.

Case study L: One firm assigned accountability as part of the design of each AI use case. Ultimate accountability sat with the SMF responsible for the business area the tool would operate in, while a separate model owner oversaw the technical build. The governance process documented both roles, giving the firm clear business and technical ownership before deployment.

Case study M: Before signing off an AI tool, one firm required the relevant SMF and business areas to approve a playbook of how they would identify and handle incidents. This covered different risks relevant to the use case and documented clear roles and processes if there was a service interruption, unexpected outlier outputs or the model underperformed against agreed thresholds. Those thresholds had red, amber and green ratings based on how significant the incident was, which informed next steps such as enhanced monitoring or temporary shutdown.

Smaller firms note: Smaller firms may have simpler governance structures, but they still need to have clear responsibilities, understand the risks and make informed decisions, they are accountable for how they use AI in their business, even where third-party providers are involved. Good practice includes named accountable individuals with clear decision ownership and sign-off, and one smaller firm mapped every material AI use case to a named senior manager, with documented risk appetite and escalation routes before deployment. Another small firm emphasised the need to record whether staff needed to review AI outputs, whether this happened, and whether they approved, overrode or escalated them, creating an audit trail showing how oversight operated in practice.

Any human oversight controls should be designed around the specific use case, the level of AI autonomy and the potential impact on consumers or markets, and effective oversight is not simply about reviewing AI outputs, but about ensuring people have the right authority, information, capability and escalation routes to challenge or intervene where needed.  

Firms in good practice carried out clear risk assessments to determine where human oversight was needed and what it should look like, ensuring that an appropriately qualified person with the expertise to spot key risks and the authority to escalate them was involved.  Good practice also included embedding human-in-the-loop controls at key design and operation stages, with staff empowered to intervene, and clear thresholds for when human intervention and escalation were needed and how AI outputs could be challenged or overridden.

An area for improvement was firms relying excessively on staff to oversee AI outputs without addressing underlying model risk or governance responsibilities, human review can mask weaknesses in the model, while unclear governance responsibilities can leave gaps in accountability, escalation and oversight.  There was a specific risk identified of human review of AI outputs becoming a box-ticking exercise rather than meaningful oversight.

Case study N: One firm used AI for affordability assessments, but staff made the decisions while the system was being established. Reviewers assessed each case without being shown the AI's proposed outcome, allowing the firm to compare independent human and AI judgements and reduce the risk that reviewers would defer automatically to the system.

Case study O: In one firm, the human reviewer role became a tick-box exercise over time. Reviewers increasingly accepted AI-generated recommendations without sufficient scrutiny and became less likely to challenge, even where there were signs of errors or limitations. Unclear escalation criteria meant that some AI outputs proceeded despite low confidence or uncertainty.

Clear risk appetite helps firms decide where they can use AI, what level of risk is acceptable and what controls they need before, during and after deployment. Effective approaches translate board-level and risk function expectations into practical thresholds, metrics and escalation routes, allowing firms to adopt AI where it is beneficial while identifying use cases that should be limited, strengthened or stopped.  

Effective boards in good practice treated AI as a strategic business issue, considering how it supported business objectives and affected consumers and markets, whether governance, accountability and operational resilience arrangements were appropriate, and using regular management information to oversee adoption, challenge decisions and remain within risk appetite.  

Firms developing agentic use cases in good practice registered their agents in a central inventory, maintaining a clear view of which agents were live, who owned them, what systems and data they could access, and what controls applied, supporting accountability, monitoring and change management.

Area for improvement: some firms had no defined risk appetite or thresholds, which limits their ability to judge acceptable outcomes and some defaulted to avoiding risks close to regulatory boundaries rather than identifying ways to manage and mitigate them, leading to unintended poor outcomes.

Case study P: One firm translated its risk appetite into red, amber and green performance thresholds for its AI tools. The thresholds were agreed between the business and model risk teams, with predefined actions where performance moved into amber or red. The firm reviewed results through a monitoring dashboard and escalated recurring concerns to a governance board.

Smaller firms note: Some small firms started with narrower, domain-specific use cases, avoiding large upfront transformation or broad use case adoption, and taking an incremental, risk-based approach. This allowed them to test governance, understand risks and build capability before scaling AI adoption, reducing the risk of widespread harm from untested systems.

Firms deploying AI safely and responsibly generally adopt AI systems and architecture that reflect:

• the sensitivity and security requirements of their use cases;
• recognise dependencies on AI models, including those provided by third parties, and integrate these into operational resilience planning;
• take steps to mitigate risks from cyber threats and over-dependence on a specific AI model or third party; and
• manage third-party relationships proactively so they are sighted on and can plan for model updates or changes in products.  

Good practice includes making considered decisions on whether to host AI models on company-owned architecture or a third-party cloud GPU, considering differences in capability, security and risk management.  Stress-testing AI tools before deployment enables firms to identify potential vulnerabilities and refine guardrails so that the product is resilient before going live.

On third-party relationships: firms can outsource AI services, but they remain responsible for them, effective approaches define responsibilities between the firm and its vendors, set appropriate contractual obligations, and build in visibility of model changes, fallback plans and exit options.  Firms may not be able to control changes made by large technology providers, but they can reduce disruption by understanding upcoming updates and planning for their impact. Some firms used contractual requirements to obtain notice of, and further information about, significant model changes, helping them test and integrate updates without disrupting services.

Areas for improvement: some firms relied on vendors' assurances without challenging or testing them, firms remain accountable for outsourced AI services, and if they do not test assurances, they may not understand model limitations, resilience dependencies or whether fallback arrangements would work in practice.  Single-platform dependence without exit options was also identified, firms did not consider alternative providers or develop plans in case the chosen model became unavailable or unsuitable, potentially leading to firms being locked in and unable to adapt to changing circumstances.

Case study Q: One firm designed an AI service so that it was not dependent on a single underlying model and considered how it could switch models if needed. It also maintained alternative server capacity in another location to help manage spikes in demand, reducing its reliance on one model or hosting route.

Case study R: One firm opened a demo of an AI product to members of staff before deployment, encouraging them to try to break it and offering a prize if they could, supplementing broader red-teaming exercises. The firm also gathered examples of how different types of consumers might use the tool and used these to refine it before deployment.

Case study S: One firm designed its AI service so that it did not depend on a single model provider, reducing concentration risk and giving the firm greater flexibility if there were issues with supplier performance or availability, or if strategic priorities changed.

Case study T: One firm built an AI-enabled product around a particular model without establishing a viable alternative or transition plan. When the model was retired, the firm had to redesign the product, it could have reduced the disruption and time taken by considering substitution and exit arrangements earlier.

The frontier is moving quickly, shifting from technology that helps us understand and decide, to technology that can increasingly act on our behalf. Agents can search across products, manage payments and investments, and transact directly with other agents, creating a new layer of autonomous technology between institutions and their customers.  But greater autonomy also changes the risk, poorly governed AI can create consumer harm, cyber vulnerabilities and operational failures, potentially at much greater speed and scale.  That is why firms need to invest now in cyber and operational resilience, and the FCA's rules already require firms to report major incidents, which must extend to significant AI failures and near-misses. As the AI Minister has recognised, near-miss reporting for agentic AI is becoming a national security priority, and it is a regulatory priority too.

Before deploying more autonomous agentic systems, firms in good practice designed a suitable governance structure, including escalation routes and compliance controls that maintained accountability, human oversight and risk management across complex workflows.  A system that is safe at deployment may behave differently at scale, under stress or as the models beneath it change, agentic systems therefore require continuous, conditional assurance.  The FCA has confirmed that it will work with industry, fellow regulators and frontier labs to develop a trusted framework for agentic finance, including clear approaches to agent identity, delegated authority, accountability and execution.

Across the FCA, AI is being deployed to scale operations and focus on the greatest harms, with AI gathering evidence, surfacing intelligence from disparate sources and flagging early warning signs so the regulator can act at pace.  The FCA is building AI tools for the oversight of 60,000 anti-money laundering (AML) firms coming into its remit in 2028, and is deploying a market first responder agent that will supervise around 9,000 small wholesale markets transaction reporting firms, so supervisors can prioritise more complex supervisory activities.  Dynamic data collection is becoming part of the future of supervision, building on the launch of the FCA's Handbook API and work on machine-readable data collection APIs.  Firms should expect the FCA's supervisory engagement to become more data-driven and targeted as these tools mature.