When IRB models were introduced by Basel II around 2004 the idea was to achieve a more risk sensitive approach to calculate capital requirements for credit risk. The Basel Committee then recognized the importance of the performance of IRB models on the one hand and of the comparability across credit institutions on the other hand.

What we have been observing, is that, increasing complexity in financial products and the individual choices of credit institutions in scope, as well as application of IRB models resulted in a high variability of risk-weighted assets.

In retrospect, many aspects which seemed unproblematic ostensibly, turned out to be fraught with problems. These issues have been addressed by regulation, which now appears more forward-looking and future-oriented than previously.
Nevertheless, a reduction in complexity must not soften the substance of current regulation, in particular, proportionality should be more pronounced. Proportionality, as applied in a correct manner, could foster a better relation between regulation and the risks of various business models.
Ideally, one would follow Einstein’s quote “Everything should be made as simple as possible, but not simpler.”

All views and opinions expressed in this interview are solely the author’s own and do not express or represent any view or opinion of Oesterreichische Nationalbank.

What are the key issues with IRB models?

Key issues of IRB models typically lie in the area of data quality and validation. Likewise, the area of low default portfolios is a common challenge, as is the appropriate mapping of collateral for credit risk mitigation.

According to the ancient Roman saying “Ex falso, quodlibet” (from falsehood anything may be derived) wrong data are a prominent source of critical annotations in the course of a model audit, making data quality an issue of paramount importance. Responsibilities concerning data should always be defined in a clear and transparent way.

Firstly, at the start of the credit approval, a designated unit, typically in the area of sales, is solely responsible for requesting and maintaining correct data of the borrower for a transparent and comprehensive representation of the borrower’s economic and legal situation in order to assess the associated credit risk. Inappropriate inputs of balance sheet data or corporate forms affect the selection of rating models and incorrect rating models might be selected for the evaluation of the borrower. Similarly, incorrect personal data of the borrower or false mapping to an industry sector may also influence the rating and ultimately result in wrong risk weights. Credit institutions often implement four-eyes principles or automated input checks (e.g. drop-down boxes) to minimize erroneous inputs. Nevertheless regular data quality and plausibility checks by the second line of defense and prudent validation are important to ensure appropriate data quality. Added to that, activities of the internal audit function (third line of defense) may also help to enhance data quality.

Secondly, data cleansing in preparation of the rating or the validation process, is an additional source of distorted and/or erroneous data. Clear and transparent rules for data exclusions are not always in place. In the event of model changes it may also be necessary to retrieve old data from the systems and fit it to current requirements. Again, transparent modification rules are needed to ensure that these modifications are adequate.

The underlying IT system is the fundament of each rating model. Data from different sources (core banking system, customer relationship system, collateral management system etc.) have to be channeled to the internal model for credit evaluation. Media disruptions is one frequent source of errors. Weakly designed interfaces or even uploads from MS Excel sheets require manual treatments and hence are prone to errors.

The rating system itself is subject to regulatory supervision and assessment. Firstly, the completeness of the rating model should ensure that all parameters relevant for proper PD estimations are considered. Secondly, the discriminatory power of the model must prove as appropriate. Thirdly, a comprehensive testing of the predictive ability should demonstrate the adequacy from various angles (e.g. on rating grades, on portfolio level etc.). Fourthly, checks on the stability of the rating model should ensure its adequate application. These four quality indicators can give rise to supervisory action, for example due to missing parameters, sparse data and changes in the credit strategy. In analogy, models for EAD, LGD and CF may show similar weaknesses due to data availability, data quality or expert judgements.

How can IRB models be best developed for low default portfolios?

You know what they say: nothing will come from nothing. Low default portfolio models have long been an issue since the variability of the results is high due to the low number of defaults. The statistics involved is challenged as are approaches for adequate validation. While an international standard has not emerged yet, literature provides some answers on how to tackle the problem of sparse data, e.g.:

–A most simple approach is to use a multi-period approach for estimating PDs and for the validation to be performed. However, even in that case statistics may be unreliable.

–Benchmarking according to Art. 180 lit. f CRR may be a practical approach to deduce proper PDs. The external PDs are subject to validation or plausibility checks by the credit institution.

–Extracting PDs from migration matrices is subject to assumptions regarding the time-homogeneity of migration matrices.

–Confidence based methods, as proposed by Pluto and Tasche in 2005, as well as derivatives from this approach may be options. Critical aspects in these approaches, however, remain the degree of conservatism which is controlled by the choice of the confidence level for estimating the upper confidence limit, the asset correlation and the year-to-year correlation.

–As suggested by Tasche in 2015, a Bayesian approach can be applied but involves expert judgement regarding the applicable prior distribution.

In any case, the choice of the model approach in combination with the relevant assumptions (correlations and confidence levels or prior distributions) will have an impact on PD estimates. This results in a significant variance of risk weighted assets which is not preferable from a supervisor’s view. Further, the validation unit is obliged to prove that the chosen method is prudent. Since backtesting is usually not a viable method for low default portfolios, alternative approaches like benchmarking need to be employed. Overall, the appropriateness and prudence of the chosen approach to handle low default portfolios as well as the consideration of model risks involved are central aspects of supervisory assessments.

What are the benefits and issues with using the standardised approached models versus internal models?

Risk comes from not knowing what you are doing, no matter which approach you adopt. Proper risk management is beneficial for borrowers, depositors and investors. An appropriate risk management system involves three steps: correct risk identification and measurement, risk management and risk bearing or risk provisioning. From this perspective an IRB approach, if applied correctly and prudent, supplies more information on the quality of loans and therefore can enhance credit decisions, which is ultimately beneficial in terms of the overall economy and financial stability. In allowing for a more precise pricing of loans, IRB models support the profitability of the credit institutions and help to keep risk capital at adequate levels. Besides, advanced portfolio management and credit strategies are impossible without detailed information on a single-loan basis.

Sure enough, these benefits come at a cost. An IRB approach requires highly developed and expensive IT systems as well as appropriate resources in terms of infrastructure, processes and personnel to fulfil the regulatory requirements in practice.

Nevertheless, regulatory expectations imply that credit institutions having business involving complex products or high exposures should have procedures and processes in place that are quite similar to those of an IRB approach. This also holds for Pillar II requirements, where processes and procedures comparable to an IRB approach should be implemented for large or complex credit businesses. However, the regulatory assessment of models applied in Pillar II is subject to the principle of proportionality, wherefore different requirements may be exerted.

How can banks deal with model risk at Pillar II for the ICAAP?

The Board of Governors of the Federal Reserve System (SR Letter 11-7) has defined model risk as “the potential indirect costs of relying on models, such as the possible adverse consequences (including financial loss) of decisions based on models that are incorrect or misused.“ and therefore concludes: “Those consequences should be addressed by active management of model risk.”

Realisations of model risk can be distinguished into two main categories: Firstly, fundamental errors may result from simplifications, approximations, shortcuts and the like. Secondly, an incorrect use results from applying the model at situations, products, or markets for which the model was not designed for in the first place.

Model risk should therefore be cared of starting with the implementation and design of the model. A clear definition of its purpose is vital for aligning the development towards the intended use. Also data and information quality should be considered, especially in cases where proxies and expert judgements are in use, a proper assessment of both, as well as their validity and limitations is mandatory. Likewise, testing is an integrated part of the implementation phase; it should cover all aspects of the model such as accuracy, stability, limitations, variations of expert judgements and assumptions, extreme values etc.

The second step concerns the model use, where a feedback loop should be established between the practical model use and its development. This should help to improve model quality and its proper application. Ideally, margins of conservatism should be applied in a way that a clear distinction between the calculated risk and the add-on due to conservatism is possible.

A validation exercise prior to the first use of a model should reveal hidden problems. The potential risk covered by the model, either due to complex instruments or high exposures, should define the range and rigor of the validation.

An ongoing validation of models, which includes all model components, i.e. input, processing and reporting, should identify arising model limitations and keep track of identified weaknesses. Therefore, an effective validation framework comprises of three core elements:
Firstly, the evaluation of conceptual soundness, which should inter alia cover all assumptions, definitions, and expert judgements.
Secondly, the monitoring of changes taking place due to a changing environment and the results of checks on data quality. Benchmarking with alternative data or models should be performed additionally.
Thirdly, thorough analyses of the model outcome and its backtesting.

A robust set-up, development, implementation, and use of models are essential to minimize model risk. However, at the level of the whole credit institution, a sound validation process to objectively assess model risk as well as costs and benefits of model changes or errors is absolutely essential.

What would you like to achieve by attending the 3rd Edition IRB Models, the Standardised Approach for Credit Risk, and Capital Floors?

The marcus evans forum is an excellent opportunity to refresh one’s ideas on the topic, while making a constructive contribution to a panel discussion. My aim is to exchange on critical thoughts concerning IRB models from an auditor‘s perspective, highlight challenges for practitioners in risk management and raise awareness for certain topics in this area which support improvement in the implementations of IRB models. Prudent IRB set-ups enhance the stability of each single credit institution and therefore the stability of the whole financial system, which apparently is in the interest of all market participants.