The number of models is rising dramatically—10 to 25 percent annually at large institutions—as banks utilize models for an ever-widening scope of decision making. More complex models are being created with advanced-analytics techniques, such as machine learning, to achieve higher performance standards. A typical large bank can now expect the number of models included within its model risk management (MRM) framework to continue to increase substantially.
Among the model types that are proliferating are those designed to meet regulatory requirements, such as capital provisioning and stress testing. But importantly, many of the new models are designed to achieve business needs, including pricing, strategic planning, and asset-liquidity management. Big data and advanced analytics are opening new areas for more sophisticated models—such as customer relationship management or anti-money laundering and fraud detection.
The promise and wider application of models have brought into focus the need for an efficient MRM function, to ensure the development and validation of high-quality models across the whole organization—eventually beyond risk itself. Financial institutions have already invested millions in developing and deploying sophisticated MRM frameworks. In analyzing these investments, we have discovered the ways that MRM is evolving and the best practices for building a systematically value-based MRM function (see sidebar, “Insights from benchmarking and MRM best practices”). This article summarizes our findings.
Model risk and regulatory scrutiny
The stakes in managing model risk have never been higher. When things go wrong, consequences can be severe. With digitization and automation, more models are being integrated into business processes, exposing institutions to greater model risk and consequent operational losses. The risk lies equally in defective models and model misuse. A defective model caused one leading financial institution to suffer losses of several hundred million dollars when a coding error distorted the flow of information from the risk model to the portfolio-optimization process. Incorrect use of models can cause as much (or greater) harm. A global bank misused a risk-hedging tool in a highly aggressive manner and, as a result, passed its value-at-risk limits for nearly a week. The bank eventually detected the risk, but because the risk model it used was inadequately governed and validated, it only adjusted control parameters rather than change its investment strategy. The consequent loss ran into the billions. Another global bank was found in violation of European banking rules and fined hundreds of millions of dollars after it misused a calculation model for counterparty-risk capital requirements.
Events like these at top institutions have focused financial-industry attention on model risk. Supervisors on both sides of the Atlantic decided that additional controls were needed and began applying specific requirements for model risk management on banks and insurers. In April 2011, the US Board of Governors of the Federal Reserve System published the Supervisory Guidance on Model Risk Management (SR 11-7). This document provided an early definition of model risk that subsequently became standard in the industry: “The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports.” SR 11-7 explicitly addresses incorrect model outputs, taking account of all errors at any point from design through implementation. It also requires that decision makers understand the limitations of a model and avoid using it in ways inconsistent with the original intent. The European Banking Authority’s Supervisory Review and Evaluation Process, meanwhile, requires that model risk be identified, mapped, tested, and reviewed. Model risk is assessed as a material risk to capital, and institutions are asked to quantify it accordingly. If the institution is unable to calculate capital needs for a specific risk, then a comprehensible lump-sum buffer must be fixed.
The potential value in mature MRM
The value of sophisticated MRM extends well beyond the satisfaction of regulatory regimes. But how can banks ensure that their MRM frameworks are capturing this value thoroughly? To find the answer, we must first look more closely at the value at stake. Effective MRM can improve an institution’s earnings through cost reduction, loss avoidance, and
capital improvement. Cost reduction and loss avoidance come mainly from increased operational and process efficiency in model development and validation, including the elimination of defective models.
Capital improvement comes mainly from the reduction of undue capital buffers and add-ons. When supervisors feel an institution’s MRM is inadequate, they request add-ons. An improved MRM function that puts regulators in a more comfortable position leads to a reduction of these penalties. (The benefit is similar to remediation for noncompliance.) Capital inefficiency is also the result of excessive modeler conservatism. To deal with uncertainty, modelers tend to make conservative assumptions at different points in the models. The assumptions and attending conservatism are often implicit and not well documented or justified. The opacity leads to haphazard application of conservatism across several components of the model and can be costly. Good MRM and proper validation increases model transparency (on model uncertainties and related assumptions) and allows for better judgments from senior management on where and how much conservatism is needed.
This approach typically leads to the levels of conservatism being presented explicitly, at precise and well-defined locations in models, in the form of overlays subject to management oversight. As a result, the total level of conservatism is usually reduced, as end users better understand model uncertainties and the dynamics of model outcomes. They can then more clearly define the most relevant mitigation strategies, including revisions of policies governing model use.
Profit and loss
With respect to improvement in profit and loss (P&L), MRM reduces rising modeling costs, addressing fragmented model ownership and processes caused by high numbers of complex models. This can save millions. At one global bank, the capital budget for models increased sevenfold in four years, rising from €7 million to €51 million. By gaining a better understanding of the model landscape, banks are able to align model investments with business risks and priorities. By reducing model risk and managing its impact, MRM can also reduce some P&L volatility. The overall effect heightens model transparency and institutional risk culture. The resources released by cost reductions can then be reallocated to high-priority decision-making models.
Systematic cost reduction can only be achieved with an end-to-end approach to MRM. Such an approach seeks to optimize and automate key modeling processes, which can reduce model-related costs by 20 to 30 percent. To take one example, banks are increasingly seeking to manage the model-validation budget, which has been rising because of larger model inventories, increasing quality and consistency requirements, and higher talent costs. A pathway has been found in the industrialization of validation processes, which use lean fundamentals and an optimized model-validation approach.
- Prioritization (savings: 30 percent). Models for validation are prioritized based on factors such as their importance in business decisions. Validation intensity is customized by model tiers to improve speed and efficiency. Likewise, model tiers are used to define the resource strategy and governance approach.
- Portfolio-management office and supporting tools (savings: 25 percent). Inefficiency can be reduced at each stage of the validation process, with predefined processes, tools, and governance mechanisms. These include development and submission standards as well as validation plans and playbooks.
- Testing and coding (savings: 25 percent). Automation of well-defined and repetitive validation tasks, such as standardized testing or model replication, can further lower costs.
The evolution toward capturing value systematically
To manage the P&L, capital, and regulatory challenges to their institutions’ advantage, leading banks are moving toward a robust MRM framework that deploys all available tools to capture efficiencies and value. The path to sophisticated model risk management is evolutionary—it can be usefully discussed as having three stages: building the elements of the foundation, implementing a robust MRM program, and capturing the value from it (Exhibit 1).
Building the foundational elements
The initial phase is mainly about setting up the basic infrastructure for model validation. This includes the policies for MRM objectives and scope, the models themselves, and the management of model risk through the model life cycle. Further policies determine model validation and annual review. Model inventory is also determined, based on the defined characteristics of the model to be captured and a process to identify all models and nonmodels used in the bank. Reports for internal and external stakeholders can then be generated from the inventory. It is important to note, however, that the industry still has no standard of what should be defined as a model. Since banks differ on this basic definition, there are large disparities in model-inventory statistics.
Governance and standards are also part of the MRM infrastructure. Two levels of governance are set up: one covering the steps of the model life cycle and one for the board and senior management. At this point, the MRM function will mainly consist of a small governance team and a team of validators. The governance team defines and maintains standards for model development, inventory, and validation. It also defines stakeholder roles, including skills, responsibilities, and the people who will fill them. The validation team conducts technical validation of the models. Most institutions build an MRM work-flow tool for the MRM processes.
Implementing a robust program
With foundational elements in place, banks can then build an MRM program that creates transparency for senior stakeholders on the model risk to the bank. Once model-development standards have been established, for example, the MRM program can be embedded across all development teams. Leading banks have created detailed templates for development, validation, and annual review, as well as online training modules for all stakeholders. They often use scorecards to monitor the evolution of model risk exposure across the institution.
A fundamental objective is to ensure high-quality, prioritized submissions. Model submissions missing key components such as data, feeder models, or monitoring plans reduce efficiency and increase delivery time. Efficiency can be meaningfully enhanced if all submissions adhere to standards before the validation process begins. Models are prioritized based on their importance to the business, outcome of prior validation, and potential for regulatory scrutiny.
Gaining efficiencies and extracting value
In the mature stage, the MRM function seeks efficiencies and value, reducing the cost of managing model risk while ensuring that models are of the highest quality. In our survey of leading financial institutions, most respondents (76 percent) identified incomplete or poor quality of model submissions as the largest barrier for their validation timelines. Model owners need to understand the models they use, as they shall be responsible for errors in decisions based on those models.
One of the best ways to improve model quality is with a center of excellence for model development, set up as an internal service provider on a pay-per-use basis. Centers of excellence enable best-practice sharing and advanced analytics across business units, capturing enterprise-wide efficiencies. The approach increases model transparency and reduces the risk of delays, as center managers apply such tools as control dashboards and checkpoints to reduce rework.
Process automation defines MRM maturity, as model development, validation, and resource management are “industrialized” (Exhibit 2). Validation is led by a project-management office setting timelines, allocating resources, and applying model-submission standards. Models are prioritized according to their importance in business decisions. An onshore “validation factory” reviews, tests, and revises models. It can be supported by an offshore group for data validation, standards tests and sensitivity analysis, initial documentation, and review of model monitoring and reporting. The industrial approach to validation ensures that models across the organization attain the highest established standards and that the greatest value is captured in their deployment.
The standards-based approach to model inventory and validation enhances transparency around model quality. Process efficiency is also monitored, as key metrics keep track of the models in validation and the time to completion. The validation work-flow system improves the model-validation factory, whose enterprise-wide reach enables efficient resource deployment, with cross-team resource sharing and a clear view of validator capabilities and model characteristics.
Consistent standards for model planning and development allow institutions to develop more accurate models with fewer resources and in less time. In our experience, up to 15 percent of MRM resources can be conserved. Similarly, streamlining the model-validation organization can save up to 25 percent in costs. With the significant regulatory spending now being demanded of institutions on both sides of the Atlantic, these savings are not only welcome but also necessary.
The contours of a mature stage of model risk management have only lately become clear. We now know where the MRM function has to go in order to create the most value amid costly and highly consequential operations. The sooner institutions get started in building value-based MRM on an enterprise-wide basis, the sooner they will be able to get ahead of the rising costs and get the most value from their models.