© Copyright 2019 marcus evans. All rights reserved.

For all enquiries regarding speaking, sponsoring and attending this conference contact:

Anastasia Zardili 

Email: anastasiaz@marcusevanscy.com
Telephone: +357 22 849 390 

9 Dimostheni Severi Street
Nicosia 1080


More Information

We would be delighted to provide you with more information on the conference agenda. Please fill in your details below and we will be in touch.

Model Risk Management in Banking

24-26 April 2019 | London, United Kingdom

Enquire Now

Why is it so important to quantify Model Risk?

Financial firm’s reliance on models continues to grow. On the other hand, there is a long history of material model related losses, including those due to CDOs on subprime mortgage-backed securities, CMS spread options during EUR curve inversion in June 2008, or JP Morgan’s “London Whale” losses in 2012, just to mention a few. As a result, many firms have been trying to quantify and mitigate Model Risk for individual trading positions and portfolios.

Model Risk is the risk arising from incorrect or inappropriate application of models, including the reliance on a set of assumptions that may not be in line with the model’s objectives or may only be valid for a limited range of foreseeable input data.

Federal Reserve / OCC SR 11-7 “Supervisory Guidance on Model Risk Management” ushered in fundamental change to the industry’s approach to Model Management, leading to recognition of Model Risk as an independent Risk Class to be managed alongside other Risk Classes (e.g. Market Risk, Credit Risk, Operational Risk, etc.). Initially this change mainly affected large firms regulated by Federal Reserve and OCC, but gradually other regulators (e.g. PRA “Model Risk Management for Stress Testing” and “ECB Guide to Internal Models”) aligned their model requirements to the major elements of SR 11-7. In some cases, regulators have adopted even more strict requirements. More and more, middle and smaller sized financial firms have been creating Model Risk Management functions, sometimes without a clear understanding how complex and expensive the process can be.

Thus identification of model limitations, together with quantification and mitigation of Model Risk, remain to be the most significant challenges of Model Risk management.

What are the current challenges within Model Risk management?

Since the introduction of SR 11-7 in 2011, significant progress has been made in the development of Model Risk management processes. This includes extension to the scope of covered models, strong requirements for the quality of model development and validation documentations, introduction of firm-wide model governance frameworks (committees, policies, etc.), establishment of model inventories and supporting model management workflows.

One of the biggest challenges, as I already mentioned, is Model Risk quantification which is required for all types of models. Among other things, it should analyze dependence of model outputs on modelling assumptions, which is usually based on benchmarking of the production model against more advanced / alternative modelling approaches. It should also cover model sensitivity to market non-observable parameters, model behaviour under extreme market conditions, etc. Thus it is quite a challenge to quantify Model Risk for an individual model, not to mention aggregating it across different model types.

Another popular current topic is model interconnectedness. There are at least two aspects to this.

The first one is how uncertainty / limitations of a model could impact the performance of other models. Clearly the most “influential” of these models include different types of feeder models, such as scenario generation models in Stress Testing or proxy treatments of input data. Another “influential” but less common case is where many of a firm’s models are derivations of a few general modelling approaches. For example, pricing models for structured products could be based on the application of a generic multi-asset framework, with some specifications (calibrations, choice of risk factors, etc.) for individual products.

Another aspect is the different types of model inconsistencies. For example, two different products may be similar when considered in a boundary case, but their models may imply quite different dynamics. Alternatively, large firms typically have dedicated trading systems for different asset classes, including all relevant hedging (usually vanilla) instruments. For example, Equity trading would need to hedge their quanto exposure with FX options, or a hybrid desk would use a variety of vanilla instruments from different asset classes. Sometimes those hedging models are not fully consistent, e.g. hybrid desks may apply more simplistic dividend treatment than Equity trading. Those situations need to be properly controlled and quantified, if required.

A necessary condition for a proper analysis of model interconnectedness is a robust model inventory, including a detailed record of the application of all model outputs. It is a huge task by itself, especially for firms with thousands of models. When all (or at least major) model connections are identified, some Model Risk analysis should be performed to analyse how model limitations could be propagated due to model interconnectedness. Due to the large volume of data and processing involved, there are some proposals to apply “Machine Learning” to this analysis, but, as far as I know, all of those attempts are still in early stages.  

Machine Learning (ML) by itself is a big and important topic. Significant resources are allocated to developing financial applications of ML approaches, especially for processing large and less-structured data sets. These applications include capital optimization, trading and investment strategies, stress testing, etc. At present there are no industry standards for model validation and performance monitoring of ML models, and even regulatory requirements are not specific and usually covered by general concepts like “effective challenge”. Undoubtedly we are going to witness active application of ML approaches to Model Risk management in future.

What are the ways to address the issue of diversity of models in Model Risk quantification?

While Model Risk quantification for different types of models requires different technical approaches, the main principles are broadly the same which include dependence on modelling assumptions, calibration quality and uncertainty, opaque model parameters uncertainty, numerical accuracy and stability, model performance under stressed conditions, etc.

Concerning Model Risk aggregation, perhaps it would be too simplistic to provide a single metric for it. An approach which uses several measures could, for example, cover losses with different confidence levels, arisen from incorrect or inappropriate model applications in excess of the current level of Model Reserves or Capital Adjustments, or model losses under different stress market conditions.

Also I am not a supporter of aggregating Model Risk across different model types. For example, aggregating the risks associated with pricing models, risk models and retail models may not be meaningful. It would be more appropriate to take large groups of models of the same type and aggregate Model Risk across the individual models in those groups. 

How does Model Risk Quantification lead to Model Risk Management?

Model Risk quantification is an important element of Model Risk management. It could be incorporated into Model Risk Appetite, performance against which is monitored via quantitative measures and qualitative statements. It is further supported by Model Performance Monitoring which is designed to identify potential model limitations outside of ongoing model validation, allowing the firm to proactively pursue mitigating actions, as appropriate.
Model Risk quantitative measures for individual models allow a firm to manage (from Model Risk prospective), for example, trading activity for a specific product or amount and type of counterparty exposure for a specific counterparty. On the other hand, well-defined Model Risk Appetite metrics should help senior management to make some strategic business decisions or prioritize some model developments if Model Risk becomes excessive.  

Finally, Model Risk quantification plays an important role in Model Risk reporting which is related to a regulatory requirement for senior management to “understand model capabilities, the model limitations, and the potential impact of model uncertainty for the most material models and the aggregate output”. In addition, Model Risk quantification measures could be supplemented by various “softer” metrics, such as current model performance failures, distribution of Model Risk ratings across all models in the firm, number of breaches raised due to model misuse, model validation progress, etc. Due to the complexity of Model Risk quantification, these “softer” metrics are often the dominant, if not the only, Model Risk reporting components in many institutions.

What would you like to achieve by attending the Model Risk Management in Banking?

The conference provides an excellent selection of presentations and panel discussions on designing governance framework for Model Risk management, quantification of Model Risk and Machine Learning applications. I look forward to learning about the latest trends to deal with those challenges.

Slava Obraztsov

Global Head of Model Risk

The need for more models increases as banks become increasingly complex, with large operations, and increasingly difficult risk types to comprehend. Ideally, models are stepping in to make this easier as effective decision making and risk management is implemented at the back of accurate models.  But what happens if the models are not correct? As evidenced in the 2012 JP Morgan scandal now referred to as “The London Whale”, bad model governance can lead a bank astray towards bad decision making (mispricing trades or taking on risk the bank cannot afford) and in turn huge losses. Ultimately banks cannot afford to not have a rigorous validation process in such a complex market as models advance and the important models are no longer only pricing models but sitting across the bank in treasury, risk and finance. 

This marcus evans conference will look at how banks can implement an industry standard for model risk management which includes a robust model governance standard, model inventory and means to quantify model risk.

Slava Obraztsov has been Global Head of Model Risk / Model Validation at Nomura since 2007.  His previous roles include Global Head of Model Validation at Bear Stearns, Senior Quantitative Model Risk Analyst at Commerzbank and Head of Risk Analytics at ANZ. He was awarded a PhD in Mathematics from Moscow State University and has held a number of academic positions at Russian and Australian universities.

Follow us on Social Media!

The challenges behind Model Risk

About the conference

Fix the following errors: