
A customer called his bank to close a credit card. He had a grievance and had made up his mind.
The AI agent handled the interaction well by every service measure. It was polite and accurate, confirmed the closure request, thanked the customer cheerfully and sounded almost pleased to see him go.
Everything about the response was correct except the one thing that mattered: the bank wanted to retain the customer.
1. The AI Got the Answer Right and the Business Outcome Wrong
The AI understood the request, but it did not understand the business objective, and that gap explains why scalable AI depends on business acumen, industry knowledge and domain expertise.
Over the last year, I have taken part in countless conversations about AI in boardrooms, client meetings and industry forums. Across these, one question keeps surfacing.
If AI makes intelligence widely accessible, what becomes the new differentiator?
After more than two decades of working with banks across digital banking, payments modernization, core banking transformation, lending and now AI-led initiatives, I have come to believe that the answer is business context.
Scalable AI transformation requires more than powerful models. It requires organizations to embed business acumen, industry knowledge and domain expertise into AI systems in a way that lets business intent, customer context and operational realities shape every decision and interaction.
Access to the largest models will matter less than the ability to turn business expertise into organizational intelligence and put it to work through AI, and the banks that build that ability first will set the pace for the rest of the industry.
2. Foundation Models Know Banking. They Don’t Know Your Bank.
One of the most important lessons emerging from enterprise AI adoption is that foundation models already understand banking, payments, lending, contact centers and trade finance in general terms.
What they do not understand is the institution in front of them, including the areas below.
- Your bank’s risk appetite
- Your customer retention strategy
- Your operating model
- Your exception handling practices
- Your governance framework
- Your business objectives
That knowledge lives in enterprise documents, operational processes, institutional memory and, most importantly, in the experience of Subject Matter Experts (SMEs).
As foundation models become more capable and easier to access, competitive advantage shifts from model intelligence to business context.
Leaders who once asked “how powerful a model is” now need to ask “how effectively their own business expertise has been embedded into it.”
3. Why Enterprise AI is Difficult to Scale
In our experience, enterprise AI initiatives often encounter three fundamental gaps, a framework popularized by Shreya Shankar and Hamel Husain in their work on evaluating LLM applications. Each one shows up in banking in a recognizable way.
- The Gulf of Specification is the gap between intended business outcomes and how they are defined or encoded into AI systems. Enterprise AI often fails when the business objective was never clearly translated into instructions, policies or evaluation criteria, even when the model itself was capable. The credit card agent in the opening story is a specification failure, because nobody told it that retention mattered.
- The Gulf of Comprehension is the gap between the people building an AI system and their understanding of how it behaves across thousands of real interactions. No team can read every conversation or decision log, and without domain experts reviewing samples of real outputs, failure patterns such as a missed grievance or a misread policy stay hidden.
- The Gulf of Generalization is the gap between AI performance in controlled environments and its ability to behave consistently across the full variety of real-world inputs. Many AI systems perform well in testing but struggle with ambiguity, exceptions and edge cases in production.
These gaps explain why technically impressive AI systems often fail to deliver consistent business outcomes at scale.
Domain Knowledge Graphs, Business Context Layers, AI Evaluation and AI Reliability Engineering help bridge them by aligning AI behavior with business objectives, operational realities and customer expectations.
4. Closing the Gaps: Where Domain Expertise Lives in Banking
Each banking domain carries its own form of expertise, and each one shows where AI needs a human expert to define what correct behavior looks like.
Payments: Scaling Experience
AI can generate workflows, process maps and test scenarios. Successful payments operations, however, depend on expertise across the areas below:
- Settlement flows
- Reconciliation
- Chargebacks and disputes
- Exception handling
- Regulatory compliance
- Multi-bank interactions
- Operational risk
The opportunity lies in embedding this expertise into AI systems and making it available at enterprise scale.
A payments SME who once reviewed scenarios manually can now teach AI systems what a complete payments journey looks like, from initiation through settlement, reconciliation and dispute, allowing that expertise to be applied consistently across thousands of transactions and interactions.
Lending: Scaling Judgment
AI can accelerate underwriting, onboarding and decision support. Lending decisions, however, are ultimately governed by the factors below:
- Risk appetite
- Credit policies
- Product strategy
- Regulatory requirements
- Business objectives
The highest value emerges when AI augments experienced lending SMEs. Their role evolves from reviewing individual decisions to defining policies, training agents and improving decision quality at scale. In lending, AI adds value by helping experienced lenders apply their judgment more consistently and across far more applications than a manual review process ever could.
Digital Banking: Understanding the Full Customer Journey
Digital Banking appears deceptively simple. An AI system can analyze clickstreams, summarize customer journeys, generate test scenarios and even recommend improvements to mobile or internet banking experiences.
Yet anyone who has worked on large-scale digital banking platforms knows that success depends on understanding a much wider set of factors.
- Customer onboarding journeys
- Authentication and access management
- Customer behavior across channels
- Mobile and web experience consistency
- Regulatory and security requirements
- Performance under peak loads
- Customer abandonment patterns
- End-to-end journey dependencies across channels
A login issue is rarely confined to the login screen, a failed fund transfer may trace back to a backend dependency several systems away, and a poor mobile experience can quietly erode retention and revenue.
Digital banking SMEs understand these relationships, including how customer journeys behave across internet banking, mobile banking, onboarding, payments, cards, CRM and core banking systems.
McKinsey’s Global Banking Annual Review 2026 points to the cost of missing this view. Banks added call centers, online banking, mobile apps and now AI services on top of the branch network instead of redesigning the operating model, and the report links that layering to weak productivity despite heavy technology spending. It also names end-to-end customer journey optimization among the core initiatives banks should be running.
As AI becomes embedded in digital channels, the larger opportunity is to build years of customer experience knowledge into AI systems that keep improving customer journeys at scale. The digital banking AI initiatives that succeed will be built around customer journeys, with individual transactions understood as steps within them.
Core Banking: Institutionalizing Knowledge
Core banking transformations remain among the most complex initiatives in banking, and the hardest problems they raise are rarely technical. Most of the difficulty lies in the five areas below:
- Product behavior
- Historical customizations
- Regulatory obligations
- Operational dependencies
- Customer journeys
For years, much of this knowledge resided with experienced practitioners. AI creates an opportunity to institutionalize that expertise and make it reusable across transformation programs, operations and decision-making. Combined with decades of accumulated institutional knowledge, AI can turn what was once tribal knowledge into intelligence the whole enterprise can use.
5. Scaling Expertise Through Domain Knowledge Graphs and Business Context Layers
One of the biggest learnings from enterprise AI adoption is that AI becomes significantly more effective when grounded in business context.
At QKTech, we are drawing on decades of expertise across Payments, Lending, Digital Banking, Core Banking, Contact Centers, Trade Finance and Treasury to build Domain Knowledge Graphs and Business Context Layers.
These intelligence layers capture:
- Customer journeys
- Business processes
- Operational rules
- Regulatory requirements
- Domain relationships
They help AI systems understand not just language, but business intent, customer context and expected outcomes alongside the language of a request. In the credit card example, a Business Context Layer would carry the bank’s retention policy, the offers an agent is authorized to make and the grievance categories that should trigger a retention conversation.
More importantly, these layers let one expert’s knowledge reach far more decisions than that expert could ever review personally.
- A Payments SME can influence thousands of payment decisions.
- A Lending SME can shape credit recommendations at scale.
- A Digital Banking SME can improve millions of customer interactions.
- A Core Banking SME can institutionalize years of operational knowledge.
In this way, Domain Knowledge Graphs and Business Context Layers institutionalize expertise and become the mechanism through which business knowledge turns into organizational intelligence.
QKTech’s AI Evaluation and Domain Knowledge Graph strategy is built around this principle of combining AI with deep domain context.
6. AI Reliability Is Where Engineering Meets Business Expertise
At QKTech, we have spent more than 25 years helping organizations improve the reliability of critical business processes and digital ecosystems across banking and financial services.
Historically, Reliability Engineering focused on ensuring that applications, integrations and customer journeys behaved predictably. Today, that responsibility extends to the reliability of AI-generated outcomes, which raises a new set of questions.
- Can an AI-powered lending assistant consistently align with credit policies?
- Can a conversational AI platform handle customer grievances appropriately?
- Can an AI-generated test suite adequately cover complex payment scenarios?
- Can AI recommendations be trusted when they influence customer experiences and business decisions?
Each of these is a business reliability question as much as a technology one, and unlike traditional software reliability, answering it requires both AI engineering and deep business expertise.
Reliability Must Be Measured Through Business Outcomes
Historically, software quality was measured through defects, performance and availability. In the AI era, reliability must also be measured through business outcomes.
An AI system may be technically accurate and still fail if it does not align with business objectives, customer expectations or regulatory requirements.
The ultimate test of AI reliability is not whether the model generated an answer. It is whether the model generated the right business outcome.
The NIST AI Risk Management Framework takes a similar view. It defines trustworthy AI through characteristics that include validity and reliability, accountability and explainability, and it asks organizations to weigh them in the context where the system is used.
AI-Native Applications Need Continuous Validation
As organizations move from experimentation to AI-native applications, deployment marks the start of the reliability work. Foundation models evolve, prompts change, agents interact with other systems and business contexts shift, and each change can alter how an AI system behaves. Banks are building several capabilities to keep pace with that change.
- Continuous AI Evaluation
- AI Observability
- LLM Validation
- LLM-as-a-Judge Frameworks
- Drift Monitoring
- Agent Performance Monitoring
However, none of these answer the most important question:
What does “good” actually look like?
Every successful enterprise AI initiative eventually needs someone to define it, and that role increasingly belongs to business leaders and domain experts who understand customer expectations, operational realities and business objectives. Research by Shankar and colleagues on LLM-assisted evaluation found that people refine their quality criteria as they grade real outputs, which means the experts who know the business must stay inside the evaluation loop.
In a bank, this work is concrete. SMEs review samples of real conversations and decisions, mark each one pass or fail and record the reason. Those judgments become the rubrics, test sets and calibration data that automated evaluators apply at scale, and the same SMEs revisit them whenever policies or products change.
SMEs Become Architects of Enterprise AI
As foundation models, agent frameworks and AI platforms mature, the role of SMEs shifts from executing work to designing intelligence, and their central task becomes teaching AI how the business works. Historically, SMEs validated outputs. In the AI era, their role becomes significantly more strategic, and the future SME will take on these new responsibilities.
- Define business objectives
- Train AI agents
- Shape decision frameworks
- Establish governance boundaries
- Create business context
- Continuously optimize outcomes
AI agents will execute the work while SMEs orchestrate it.
This shift transforms domain expertise from an individual capability into an enterprise asset. Organizations that successfully capture and scale it will build a significant competitive advantage.
SMEs also set the reliability target and the error budget: how many wrong outcomes the bank can tolerate, and what happens when that budget is exhausted. Deciding that number requires knowing the customer impact, the regulatory exposure and the cost of a wrong lending or payment decision, which makes it a business judgment more than a technical one.
7. The Real Competitive Advantage
The future will certainly require better foundation models, richer observability, stronger evaluation frameworks and more sophisticated AI engineering. It will equally require organizations that can capture, institutionalize and scale business expertise. As AI generates more of the answers, banks will compete on the business context they build into those answers.
Leaders can start by naming an accountable SME owner for every AI use case in production or pilot. Protect a share of that expert’s time for defining evaluation criteria and reviewing outputs, and treat it as core work. Make business outcome measures such as retention, dispute resolution and policy adherence part of the release criteria for every AI system.
Which brings me back to that credit card call. An agent taught by the bank’s own SMEs would have recognized the grievance, understood the retention objective and known exactly what the bank was willing to offer to keep the customer. The same model, handling the same request, would have produced a completely different business outcome.
Foundation models power AI and engineering scales it, but the organizations that embed business acumen, industry knowledge and domain expertise into AI systems will be best positioned to translate AI innovation into business outcomes and accelerate transformation.
Models will keep getting more capable. A bank’s own context is the one input no competitor can download.
Talk to our banking team about turning banking SME expertise into scalable AI.
With thanks to Shiladitya Roy Chaudhury and Toral Sanghavi, whose feedback shaped the structure, theme and flow of this article.
References
- Shreya Shankar and Hamel Husain, “AI Evals for Engineers and PMs” course, three-gulfs model. How to think about evals (course session notes).
- McKinsey & Company, “Global Banking Annual Review 2026: Precision with Speed,” preview, May 2026. Report page · PDF
- National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1, January 2023. Framework page · PDF
- Shankar, S., Zamfirescu-Pereira, J.D., Hartmann, B., Parameswaran, A. and Arawjo, I., “Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences,” UIST 2024. arXiv
- Beyer, B. et al. (eds.), “Site Reliability Engineering,” Google/O’Reilly, Chapter 3: Embracing Risk. Read online



